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A Guide to Reading This Web-based Paper 


When we read a paper we tend to read it in a linear way, from start to finish. However, due to the many 
complex variables and relationships pertaining to fourth-graders' writing performance being examined 
here, one should not be restricted to review this paper in such a way. 

The conceptual model below, which is located on page ix of the Executive Summary, illustrates the 
complexities of these variables and relationships. 

There are different ways to see how the categories within each variable are related to writing 
performance as well as to other variables. Here is a specific example of how one may read this paper in 
a non-linear way: 


Demographic 

Variables 



Prior Exposure to Writing on Computer 





Writing Performance 


• Look at the conceptual model on page vii in the Executive Summary. 

• Figure out which relationships you want to examine— for instance, relationship 1 (text length 
and writing performance). 

• Click on the number indicating relationship 1 in the conceptual model and it will take you to 
Executive Summary 1.1. 
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• Now look at Executive Summary 1.1, which provides a brief blurb and will direct you to links to 
the three tables (tables 3-1, 3-2, and 3-3) related to this hypothesized relationship. 

• Now click on tables 3-1, 3-2, and 3-3 to see the actual tables with a brief description of the 
findings in pages 21 and 22. 

• You may now look at Hypothesis 1 (pertaining to relationship 1) on page xi of the Executive 
Summary to see if the findings confirm or disconfirm the hypothesis. 


This pattern repeats itself for the other relationships. There are ten editing tools broken into three 
categories— language tools like spellcheck, emphasis tools like bold and revisions tools like copy. And, 
there are eight contextual variables— having to do with assignments, access at home, instruction, 
preference, and so on— that we had handpicked as being related to writing on the computer. 


If you are primarily interested in the findings regarding writing performance, click on the box titled 
"Writing Performance" and it will take you to Chapter 2. 
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Executive Summary 

Introduction 

The National Assessment of Educational Progress (NAEP) conducted a computer-based pilot writing 
assessment at grade 4 in 2012, based on the 2011 NAEP writing framework. A key question that this 
pilot assessment was designed to answer was whether or not fourth-graders could fully demonstrate 
their writing ability on a computer. 

To address this question, the National Center for Education Statistics (NCES) conducted the present 
study. Approximately 10,400 fourth-graders from about 510 schools (420 public and 90 private) 
composed responses to writing tasks reflecting three purposes of writing: to persuade or change the 
reader's point of view, to explain or broaden the reader's understanding of a topic, and to convey an 
experience— real or imaginary. Fulfillment of these purposes of writing required students to bring forth 
target skills related to language facility, development of ideas, and organization of ideas. 

The schools and students participating in this pilot assessment were selected to be representative of all 
schools nationally. Schools with high concentrations of students from certain racial/ethnic groups were 
oversampled and weighted to account for the disproportional representation of the selected sample. 

Students were randomly assigned two writing tasks (out of a total of 36) and had 30 minutes to 
complete each one. Writing tasks were presented to students in a variety of ways including text, audio, 
video, and photographs. Before being presented with the tasks, students were shown a tutorial to 
familiarize them with the way material is presented on the computer screen and show them how to use 
the custom-developed software program, which is similar to common word processing programs. 
Students' responses were scored from 1 to 6 across the three target skills, and the two scores were 
averaged for a final score. 

Data from the pilot assessment were used to test the validity of a conceptual model that identified a 
variety of factors that might be related to students' writing performance on the computer at the fourth- 
grade level. As a pilot assessment, the data were not weighted to adjust for nonresponses and this 
disallowed accurate estimation of standard errors and, consequently, statistical significance tests. In lieu 
of statistical significance tests, the findings summarized in this report represent "substantive" results, 
which are defined as 7 percentage points or higher for percentage differences, 1 effect sizes of greater 


1 A 7-percentage-point difference was applied for percentages ranging from 15 to 85. When either of the 
percentages to be compared was out of this range, a 5-percentage-point difference was used as a criterion for a 
substantive difference (Rosenthal 1996). 
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than 0.2 for score or other numerical differences, and correlation coefficients equal to or greater 
than .10, which indicate a small effect size, respectively 2 . 

Similarly, this report provides analyses at three performance levels: low representing the bottom 20 
percent, middle representing the middle 60 percent, and high representing the top 20 percent. The 
analyses describe the amount of text that fourth-grade students produced— as well as the extent to 
which they used common word processing editing tools— as they composed their responses on the 
computer. 

As shown by the findings reported below, a complex set of relationships was examined. All relationships 
were reported— substantive relationships in the body of the report and nonsubstantive relationships in 
the appendixes— for discussion and reference. Because at this point in time the topic of computer-based 
writing assessments is on the cutting edge and the data are tentative, the nonsubstantive relationships 
may be useful as well. 

In addition to the 2012 NAEP grade 4 pilot writing assessment data, this report also refers to the findings 
of a small-scale usability study conducted in 2011, as well as the 2010 NAEP grade 4 paper-based pilot 
writing assessment and the 2011 NAEP grade 8 and 12 computer-based writing assessments. 

Can fourth-graders fully demonstrate their writing ability on a 
computer? 

This study began with the policy question "Can fourth-graders fully display their writing ability on the 
computer ?" There is no simple "yes" or "no" answer to this question. While high-performing fourth- 
graders are able to fully display their writing abilities on the computer, low- and middle-performing 
fourth-graders may not be able to— based on an analysis of 15 tasks common to the 2010 pilot writing 
assessment on paper and the 2012 pilot writing assessment on the computer that aimed to minimize 
the effects of confounding variables, such as task difficulty and accessibility. 

Overall writing performance 

• More than two-thirds of fourth-graders' responses (68 percent) on the 2012 NAEP computer- 
based writing pilot assessment received scores in the bottom half of the 1 to 6 point scoring 
scale (1, 2, or 3) ( figure 2) . The average p-value 3 was .40. 

• The overall average score of fourth-graders on the 2012 writing pilot assessment was about 3. 


2 In interpreting effect sizes, the following rules were used: for Cohen's (1988) d, an effect size of 0.2 is small, 0.5 is 
medium, and 0.8 is large. As for correlations, an r of .10 is small, .30 is medium, and .50 is large (NCES 2012). 

3 In this report, p-value is solely a measure of item difficulty expressed on a 0 to 1 scale. Because no statistical 
tests were conducted for this report, p-value does not refer to the estimated probability of rejecting the null 
hypothesis when the hypothesis is true. 
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• A higher percentage of fourth-grade students in 2012 (40 percent) scored at the low end of the 
distribution (1 or 2) compared to eighth-grade students in the 2011 computer-based writing 
assessment (20 percent). 

• Average p-values for writing on paper in 2010 and on the computer in 2012 were similar— .37 
and .40, respectively. 

Writing performance on tasks common to the paper- and computer-based 
assessments 

• The percentage of responses in the top two categories of the 6-point scoring scale was higher on 
the computer (15 percent) than on the paper assessment (10 percent) ( figure 3 ). 

• High-performing students— i.e., those in the top 20 percent— scored substantively higher on the 
computer than on paper (effect size = 0.56). Low-performing students (representing the bottom 
20 percent) as well as middle-performing students (representing the middle 60 percent) did not 
appear to benefit from using the computer, with effect sizes of 0.05 and 0.16, respectively; thus, 
the use of the computer appeared to widen the achievement gap ( table 2-1 ). 

• Average scores for writing on paper in 2010 and on the computer in 2012 were similar— 2.98 
and 3.08, respectively ( table 2-1 ). 

• Average p-values for writing on paper in 2010 and on the computer in 2012 were similar— .40 
and .41, respectively— i.e., closer than they were when comparing all 36 writing tasks (.37 on 
paper and .40 on the computer). 

Text length 

• The average number of words produced by fourth-graders was smaller on the 2012 computer- 
based pilot assessment than on the 2010 paper-based pilot assessment (110 vs. 159). In 
contrast, the average number of words produced by eight- and twelfth-graders was larger on 
the 2011 computer-based assessment than on the 2007 paper-based assessment (300 vs. 155 
for grade 8; 389 vs. 235 for grade 12) ( figure 4) . 

• Low-performing fourth-graders produced fewer words (60) than middle- and high-performing 
fourth-graders (104 and 179, respectively) in the computer-based assessment ( table 3-2) . 

Typing speed 

• Approximately 29 percent of the sampled fourth-grade students in a usability study typed fewer 
than 10 words per minute. Their average typing speed was much lower than eighth-graders (12 
vs. 30 words per minute) ( figure 5 ). 

• Some sampled fourth-graders in the usability study typed fewer than five words per minute 
( figure 6) . 
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What factors are related to fourth-grade students' writing 
performance on the computer? 

To help explain the above findings, we developed a conceptual framework that identified a variety of 
contextual and demographic variables that might be related to the differential performance of fourth- 
grade students on the NAEP computer-based writing pilot assessment. This framework claimed that in 
addition to the skills required by the NAEP writing task demands outlined in the framework (language 
facility, development of ideas, and organization of ideas), there are other required skills we call 
"facilitative". This latter set of required skills includes keyboarding skills (affecting text length) and the 
use of word processing editing tools (affecting changes in words and sentences). 

To demonstrate the relationships between these facilitative skills and writing performance, a series of 
relationships were hypothesized based, in part, on a meta-analysis of recent studies conducted by 
Goldberg, Russell, and Cook (2003). The hypothesized relationships are shown in figure A. While the 
arrows in the figure indicate how the study conceptualizes the relationship between writing 
performance and various relevant factors, in the absence of controlled experimental studies, the 
relationships in and of themselves do not imply any causation. 

Figure A. Conceptual model of the relationship between fourth-graders' writing performance and various 
associated factors 


Demographic 

Variables 
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Prior Exposure to Writing on Computer 
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Hypotheses 

Hypothesis 1: Text length and the use of editing tools will affect writing performance (addressing 
Relationships 1 and 2). 

Hypothesis 2: Text length and the use of editing tools have a type of reciprocal relationship:— if more 
text is produced, a student will have more opportunity or need to use editing tools to improve the 
quality of the writing, and the use of these tools will free cognitive resources that can be devoted to 
developing ideas and, consequently, result in longer texts (addressing relationship 3). 

Hypothesis 3: Prior experience with writing on the computer will affect the amount of text produced 
and the extent to which editing tools are used (addressing relationships 4 and 5). 

Hypothesis 4: Prior exposure with writing on the computer directly affects writing performance 
(addressing relationship 6). 

Hypothesis 5: Prior experience with writing on the computer will also affect students' preference for 
mode of writing (computer vs. paper) and, consequently, text length, their use of editing tools, and 
writing performance. This is likely because students who experience less performance anxiety and 
greater self-confidence when writing on the computer may put forth more effort and persistence when 
faced with a challenge (addressing relationships 7 to 11). 

Hypothesis 6: Certain demographic groups will have less exposure to writing on the computer 
(addressing relationship 12). 

The Findings 

The empirical correlations observed between performance and the contextual and demographic factors 
largely supported the predictions as specified in the conceptual model, including the key prediction that 
the differential effects of the computer on the writing performance of high- and non-high-performing 
fourth-graders would be related to their prior exposure to writing on the computer. The data indicate 
that prior exposure (e.g., having access to the Internet at home) is associated with the following: (a) text 
length (which is related to keyboarding skills); (b) uses of editing tools such as the spellcheck and 
backspace keys (which are related to editing words and sentences); (c) preference for mode of writing 
(computer vs. paper); and (d) certain demographic characteristics— all of which are related to writing 
performance. Furthermore, the data show that prior exposure to writing on the computer is itself 
associated with writing performance. 

The substantive findings supporting the above hypotheses are outlined below. Note that the study's 
substantive findings are discussed in the body of the report, whereas the nonsubstantive findings can be 
found in appendixes C to E. While the nonsubstantive findings do not meet the statistical standards of 
the substantive findings defined above, they are, nevertheless, informative in identifying topics for 
future research. 
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I. Relationships among text length , use of editing tools , and writing 
performance 

1. The relationship between text length and writing performance 
(Hypothesis 1) 

• The longer a student's response, the higher a score it was likely to receive ( table 3-1) . 

• Compared to middle- and high-performing students, low-performing students, on average4, 

o generated fewer words ( table 3-2 ). 
o used fewer key presses ( table 3-3) . 

2. The relationship between the use of editing tools and writing 
performance (Hypothesis 1) 

• Compared to middle- and high-performing students, a higher percentage of low-performing 
students 

o did not use spellcheck ( table 3-4 ). 

o did not accept any automated spelling corrections ( table 3-4 ). 
o used the backspace key less frequently ( table 3-5 ). 

• A higher percentage of low-performing students used cut and copy tools ( table 3-5 ). 

3. The relationship between text length and the use of editing tools 
(Hypothesis 2) 

• Fourth-graders were likely to write longer responses if they 

o accepted automated spelling corrections more often ( table 3-6) . 
o used the backspace key more often ( table 3-7) . 

• Fourth-graders who did not use the cut revision tool tended to write longer responses than 
those who used these tools 1 or more times ( table 3-7) . 


Note that scores or word counts comparisons are always based on the average values of two groups. To avoid 
wordy statements, the term "on average" is not used all the time. 
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II. Relationship between text length , use of editing tools , writing 
performance ; and prior exposure to writing on the computer 

1. The relationship between text length and prior exposure (Hypothesis 3) 

• Fourth-graders were likely to write longer responses if 

o they had Internet access at home ( table 4-1) 
o they used the Internet to look for information for writing ( table 4-2 ) 
o they wrote to friends and family using the Internet ( table 4-3 ) 

2. The relationship between editing tools and prior exposure (Hypothesis 3) 

• Fourth-graders who never or hardly ever received computer-based writing assignments in a 
school year were more likely than those who did at least once a week 

o not to accept automated spelling corrections ( table 4-4 ). 
o to use the backspace less often ( table 4-5 ). 

• Fourth-graders who did not have access to the Internet at home were more likely than 
those who had access 

o to use spellcheck tools less often ( table 4-6 ). 
o not to use the thesaurus ( table 4-7) . 
o not to use bold and italics ( table 4-8) . 

• Fourth-graders who never or hardly ever looked for information on the Internet to include 
in their writing were more likely than those who did so at least once a week 

o to accept automated spelling corrections less often ( table 4-9 ). 
o to use the backspace key less often ( table 4-10 ). 

• Fourth-graders who never or hardly ever wrote to friends or family using the Internet were 
more likely than those who did so at least once a week 

o to accept automated spelling corrections less often ( table 4-11) . 

• Fourth-graders who never or hardly ever wrote to friends or family using the Internet were 
more likely than those who wrote more often 

o to use the backspace key less often ( table 4-12 ). 
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3. The relationship between writing performance and prior exposure 

(Hypothesis 4) 

• The 12 percent of fourth-graders who never or hardly ever received instruction in 
keyboarding and word processing had a lower average score on the assessment than those 
who did ( table 4-14 ). 

• The 31 percent of fourth-graders who never or hardly ever received computer-based writing 
assignments had a lower average score on the assessment compared to those who received 
such assignment ( table 4-15) . 

• The 23 percent of fourth-graders who had no access to the Internet at home had a lower 
average score than those who had ( table 4-17 ). 

• The 32 percent of fourth-graders who had never or hardly ever searched the Internet for 
information to include in their writing had a lower average score than those who did so once 
or twice a month or a few times a year ( table 4-18 ). 

• Higher percentage of low- and middle-performing students (44 percent and 46 percent) 
never or hardly ever used the Internet to write to friends or family compared to high- 
performing students (35 percent) ( table 4-19 ). 

• Across performance levels, students who spent at least an hour on the computer for school 
assignments in a day had a lower average score than those who spent less ( table 4-20) . 

III. Relationship between text length , use of editing tools , prior 
exposure to writing on the computer , performance , demographic 
variables and preference for mode of writing 

1. The relationship between text length and preference for mode of writing 

(Hypothesis 5) 

• Fourth-graders who preferred to write on paper 

o wrote shorter texts on average ( table 5-1 ). 
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2. The relationship between editing tools and preference for mode of 
writing (Hypothesis 5) 

• Fourth-graders who preferred to write on paper were more likely to 

o use spellcheck tools less often ( table 5-2 ). 
o use thesaurus less often ( table 5-3 ). 
o use bold or Italic tool less often ( table 5-4 ). 
o use backspace less often ( table 5-5) . 

3. The relationship between prior exposure and preference for mode of 
writing (Hypothesis 5) 

• Fourth-graders were more likely to say that they preferred to write on paper if they 

o had no access to the Internet at home ( table 5-6) . 

o had little or no experience writing to friends and family using the Internet. 

(compared to those who did so at least once a week) ( table 5-7) . 
o spent no time writing on computer for school assignments ( table 5-8 ). 

4. The relationship between performance and preference for mode of 
writing (Hypothesis 5) 

• Fourth-graders who preferred to write on paper 

o had a lower average score ( table 5-9) . 

5. The relationship between demographic variables and preference for 
mode of writing (Hypothesis 5) 

• A higher percentage of fourth-graders with the following characteristics preferred to write 
on paper: 

o male, English Language Learners (ELL), and students with a disability ( table 5-10 ). 

IV. Relationship between prior exposure to writing on the computer 
and demographic variables (Hypothesis 6) 

• A higher percentage of Black and Hispanic students than Asian students did not receive 
instruction in keyboarding and word processing ( table 6-1 ). 
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• The percentage of fourth-graders who never or hardly ever received computer-based 
writing assignments was higher among ELL, National Student Lunch Program (NSLP)-eligible, 
and Hispanic students ( table 6-2 ). 

• The percentage of fourth-graders who never or hardly ever took a computer-based writing 
test with an extended constructed-response component was disproportionately higher for 
Hispanic students and ELL students, as compared to their counterparts, i.e., non-Hispanic 
students (including white, black and Asian students), and non-ELL students, respectively 
( table 6-3) . 

• The percentage of fourth-graders who did not have access to the Internet at home was 
higher for Black and Hispanic students than White or Asian students, NSLP-eligible students 
than NSLP not-eligible students, ELL students than non-ELL students, and students with a 
disability, compared to students with no disability ( table 6-4 ). 

• The percentage of fourth-graders who looked for information on the Internet to include in 
their writing at least once a week was lower for White students compared to Black, 

Hispanic, or Asian students ( table 6-5) . 

• A lower percentage of female than male students reported having never or hardly ever 
written to friends and family using the Internet ( table 6-6 ). 

• The percentage of fourth-graders who had no experience of writing on the computer for a 
school assignment was higher for White students compared to Black, Hispanic, or Asian 
students, ( table 6-7) . 
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Abstract 


This study examined whether or not fourth-graders could fully demonstrate their writing skills on 
the computer and factors associated with their performance on the National Assessment of 
Educational Progress (NAEP) computer-based writing assessment. The results suggest that high- 
performing fourth-graders (those who scored in the upper 20 percent in the computer- and paper- 
based writing assessments on the NAEP) write better on the computer than on paper and can fully 
demonstrate their writing proficiency in a computer-based assessment. 

There are, however, indications that low-performing fourth-graders (i.e., those in the bottom 20 
percent) and middle-performing fourth-graders (representing the middle 60 percent) may have 
performed less well on the computer than on paper. This suggests a potential differential effect of 
writing on the computer on writing performance that might have contributed to the widened 
achievement gap between high- and non-high-performing (i.e., low- and middle-performing) students 
in the 2012 computer-based pilot assessment, compared to the 2010 paper-based pilot assessment. 

Unequal prior exposure to writing on the computer and preference for mode of writing (paper vs. 
computer) appear to be associated with this widening of the achievement gap. For example, about 94 
percent of high-performing students in 2012 reported having access to the Internet at home, 
compared to about half (52 percent) of low-performing students. The score difference between those 
with access to the Internet at home and those without was substantive, with an effect size of 0.87. 

The statistics presented in this report are estimates based on a NAEP pilot assessment conducted with 
a nationally representative sample of 10,400 grade 4 students. As a pilot assessment, the sample was 
not adjusted to account for nonresponse bias. As a result, the sample may not completely represent 
the grade 4 population. 
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Chapter 1. Introduction 

1. Background 


In 2012, the National Center for Education Statistics (NCES) administered the National Assessment of 
Educational Progress (NAEP) computer-based pilot writing assessment at grade 4. The key question that 
the pilot assessment was designed to answer was whether fourth-graders, especially low-performing 
students, could fully demonstrate their writing ability on a computer. Since this was a pilot assessment, 
no scale scores based on the item response theory (IRT) were produced. Instead, writing performance 
was reported using a 6-point scoring scale, where 1 represents the low end of performance and 6 
represents the highest performance. Each sampled student was given two 30-minute tasks to complete 
and received a score that was computed as an average of their scores on the two tasks. The overall 
average score was about 3 points. 

Students used common word processing tools to compose and edit their responses to the writing 
assessment tasks. This NCES report provides the results of analyses that describe the length of their 
responses to these tasks as well as the frequency with which they used the editing tools. The sampled 
students were divided into three performance levels as follows: 

• Low: This level corresponds to those in the bottom 20 percent. The average score for students at 
this level is 1.5, which, according to the scoring rubric description, indicates writing skills in a low 
or marginal state. 

• Middle: This level corresponds to those in the middle 60 percent. The average score for students 
at this level is 3.0, which indicates writing skills in a developing state. 

• High: This level corresponds to those in the top 20 percent. The average score for students at 
this level is 4.8, which indicates close to competent writing skills. 

2. Conceptual framework: overview 

Before conducting the analyses for this study, it was first necessary to develop a conceptual framework 
of the relationships that exist between fourth-graders' writing performance, their keyboarding skills and 
use of editing tools (referred to here asfacilitative skills), and relevant contextual factors. 

The diagram in figure 1 displays writing performance as a dependent variable, with three associated 
factors— text length, preference for mode of writing (paper vs. computer), and the use of editing tools. 
Prior exposure to writing on the computer is assumed to be positively correlated with all three factors. 

In this diagram, the direction of the arrows indicates how this study conceptualizes the relationship of 
writing performance with various relevant factors that act as predictors, but not as causal factors. 
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Figure 1. Conceptual model of the relationship between fourth-graders' writing performance and various 
associated factors 


Demographic 

Variables 



Prior Exposure to Writing on Computer 



Writing Performance 


According to the model, both text length and the use of editing tools are predictors for writing 
performance. But text length and the use of editing tools themselves have a type of reciprocal 
relationship— if more text is produced, a student will have more opportunity or need to use editing tools 
to improve the quality of the writing, and the use of these tools will free cognitive resources that can be 
devoted to developing ideas and, consequently, result in longer texts. 

Prior experience with writing on the computer might be posited as the primary "predicting" factor, as it 
could directly affect the amount of text produced and the extent to which editing tools are used. 
However, prior experience might also have a variety of indirect effects on writing performance. As an 
example, a student with prior experience might experience less performance anxiety and greater self- 
confidence, because he or she is more facile with working in a computer task environment; hence the 
student would prefer to write on the computer in an assessment. Fourth-grade students' preference for 
mode of writing may also affect the effort they put into writing on the computer and their persistence 
when faced with a challenge, which can have an impact on their writing performance. 

To test these hypotheses, it was necessary to develop an expanded view of word processing in relation 
to quantity of writing and quality of writing. All available contextual variables related to the NAEP 
fourth-graders' use of the computer— i.e., keyboarding skills, the use of editing tools, preference for 
mode of writing, and demographic characteristics— were examined. 
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This report represents the full set of results of the analysis of the 2012 NAEP computer-based writing 
pilot assessment administered at grade 4— from an examination of the relationship between students' 
word processing skills and writing scores to an exploration of how these relationships vary according to 
a host of factors, such as students' performance level, access to the Internet at home, and keyboarding 
experience, as well as demographic characteristics, such as race/ethnicity, gender, student disability (SD) 
status, English language learners (ELL) status, and National School Lunch Program (NSLP) eligibility. 

3. Conceptual framework: facilitative skills 

It is important to note that the NAEP writing construct is no longer "writing," but rather "writing on the 
computer". This means that in addition to the target skills measured by the NAEP Framework— i.e., 
language facility, development of ideas, and organization of ideas— writing requires skills related to 
keyboarding and word processing. These skills are assumed, but not directly measured, by NAEP. That is, 
the NAEP writing tasks and scoring rubrics assess students only in the target domain and do not take 
into account deficiencies in facilitative skills in the same way that they do not, for example, take into 
consideration the reading requirements of the prompt itself 5 . Because facilitative skills are assumed, 
the tutorial given prior to the assessment does not train students in how to use the "common" editing 
tools— although it introduces them to the tools. In other words, NAEP does not mitigate the effects of 
facilitative skill variation via task design, scoring design, or the tutorial because these skills are not 
perceived to be part of the target domain (National Assessment Governing Board 2010). 

Nonetheless, because keyboarding skills and the use of editing tools— unlike using a pencil— are 
believed to impact writing performance (e.g., Goldberg, Russell, and Cook 2003), it is necessary to study 
these skills and their relationship to relevant prior experiences, such as keyboarding experience, in order 
to develop a more valid measure of the target skills (McLaughlin 2008). Because mastering the required 
target skills necessitates some level of keyboarding and word processing skills, this, in turn, necessitated 
a new conceptual framework for the facilitative skills. 

Earlier studies on the impact of word processing on the quantity and quality of students' writing are 
generally outdated. In addition, these studies do not take into consideration student-level 
characteristics, such as students' performance level. Regardless of these obstacles, however, a meta- 
analysis by Goldberg, Russell, and Cook (2003) of articles published between 1992 and 2002 provides 
some evidence of the positive overall effects of word processing on the quantity and quality of student 
writing at the middle and high school (but less so at the elementary) levels. Moreover, they suggest that 
revisions made by students who use word processing software result in higher quality writing than 
revisions made by students who write on paper. These findings were helpful in the development of the 
conceptual framework presented in figure 1. 


5 The term "prompt" refers to the text of the assessment question. The term "assessment task," on the other 
hand, refers to the prompt plus its associated stimuli. For the purposes of this report, they are used 
interchangeably. 
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4. Conceptual framework: editing tools 

The use of common editing tools provided by NAEP's word processing software, which is comparable to 
Microsoft's, is perceived to be informative of a student's cognition as it relates to the framework's three 
writing demands— language facility, development of ideas, and organization of ideas— more so than is 
the use of general computer or interface tools, such as collapsing or expanding prompts, scrolling, and 
zooming. 

More specifically, as shown below, the use of the thesaurus and spellcheck tools tend to be relevant to 
the language demand of the NAEP writing framework— specifically, vocabulary. The use of bold, italics, 
and underline (used primarily for emphasis)— as well as of backspace, cut, copy, paste, and delete (used 
primarily for revision)— tend to be most relevant to the development and organization of ideas. 

Language tools 

Students use these tools to correct misspellings and to improve their vocabulary with appropriate 
alternatives: 

• Spellcheck 

• Thesaurus 

Emphasis tools 

Students use these tools to emphasize specific words or sections of their responses: 

• Bold 

• Italic 

• Underline 

Revision tools 

Students use these tools to make changes to their responses: 

• Backspace (located on the keyboard) 

• Delete (located on the keyboard) 

• Cut 

• Copy 

• Paste 

Note that the NAEP word processing software did not provide a grammar checker because of the on- 
demand, first-draft nature of the writing tasks, not because grammar is unimportant. However, it is 
expected that fourth-graders will make both grammatical and spelling errors. As long as these errors are 
not so pervasive as to impede the understanding of a text, they are not considered relevant to 
evaluating the quality of students' responses, even at high score-point levels. Similarly, NAEP did not 
provide tools that could potentially become distractions, such as the ability to change font and color. 
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5. Defining the editing tools 

Spellcheck 

Spellcheck can be accessed in any of the following three ways: (a) clicking on tools in the menu bar, (b) 
using the spellcheck icon from the toolbar, and (c) right-clicking anywhere in the response text- 
followed by clicking check spelling from a pop-up menu. Once spellcheck has been accessed, it identifies 
misspelled words in red (one at a time in the spellcheck dialogue box) and provides a list of suggested 
words for students to consider and substitute. At this time— independent of the method used to access 
spellcheck— the computer captures every time the student accepts a suggested word by clicking on 
"change" (but not if the student chooses, "ignore" or "ignore all"). These captured actions are referred 
to Accepted Automated Spelling Corrections in this study. 

Thesaurus 

The thesaurus can be accessed in any of the following three ways: the menu bar, the toolbar, and right- 
clicking in the response text area. Once the thesaurus is accessed, it identifies a list of suggested 
alternatives. At this time— independent of the method used to access the thesaurus— the computer 
captures Thesaurus Replacement— i.e., every time the student decides to replace a word with a 
synonym or antonym offered by the thesaurus. 

Bold, italic, underline 

The computer records every time the student uses the bold, italic, or underline tools. Students were 
shown— but not trained in during the tutorial— four different ways to access these tools: (a) the B, /, and 
U buttons on the toolbar; (b) the keystroke combinations Ctrl + B, Ctrl + I, and Ctrl + U; (c) Format 
followed by Bold, Italic, or Underline on the menu bar; and (d) right-clicking in the response text area 
and selecting Bold, Italic, or Underline from a pop-up menu. The computer software, in its current 
design, does not differentiate between adding and removing these formatting commands, counting 
each as a use. 

Backspace, delete, cut, copy, and paste 

The computer records every time the student presses the backspace and delete keys on the keyboard in 
a writing response. The difference between these keys is simply in their positioning and direction. When 
a student presses the backspace key, characters are deleted to the left of the cursor position. When a 
student presses the delete key, characters are deleted to the right of the cursor. 

Students were given four different ways to access the cut, copy, and paste tools: (a) the "scissor" icon, 
the "paper" icon, and the "clipboard" icon on the toolbar; (b) the keystroke combinations Ctrl + X, Ctrl + 
C, and Ctrl + V; (c) Edit followed by Cut, Copy, or Paste on the menu bar; and (d) right-clicking in the 
response text area and selecting Cut, Copy, or Paste from a pop-up menu. 
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6. Defining the questionnaire variables 

Data from the 2012 NAEP writing assessment background questionnaires, which were completed by 
fourth-grade students and school administrators, can be used to examine what student and school 
factors are related to writing performance. These factors can be used to inform policymakers', 
educators', and assessment developers' understanding of how to improve students' writing skills on the 
computer and build a better measure of these skills. 

The data on text length and frequency of editing tool use— central to the policy question at hand— are 
related to eight contextual questions on the topics listed below. Three of the questions are from the 
school questionnaire, which is distinct from the teacher questionnaire (see Cautions in Interpretation 
below). The other five questions are from the student questionnaire. (The exact texts of the questions 
are given in appendix A.) 

School questionnaire 

1. Instruction in keyboarding and word processing 

2. Computer-based writing assignments 

3. Computer-based writing tests with an extended constructed-response component 

Student questionnaire 

1. Internet access at home 

2. Using the Internet to get information for writing 

3. Using the Internet to write to friends or family 

4. Computer-based writing assignments 

5. Preference for taking a writing test (on computer, on paper, it does not matter) 

7 . Research questions 

By chapter, the research questions examined in this report are as follows: 

Chapter 2 

• Can fourth-grade students fully display their writing skills on the computer? 

Chapter 3 

• What is the relationship between fourth-graders' writing performance and their (a) text length 
and (b) use of editing tools? 

• What is the relationship between fourth-graders' text length and their use of editing tools? 
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Chapter 4 

• What is the relationship between fourth-graders' prior exposure to writing on the computer and 
their (a) text length, (b) use of editing tools, and (c) writing performance? 

Chapter 5 

• What is the relationship between fourth-graders' preference for mode of writing and (a) text 
length, (b) use of editing tools, (c) prior exposure to writing on the computer, (d) writing 
performance, and (e) demographic characteristics? 

Chapter 6 

• What is the relationship between fourth-graders' prior exposure to writing on the computer and 
their demographic characteristics? 

8. Methods 

This report presents analyses and findings for the 2012 NAEP grade 4 pilot writing computer-based 
assessment (WCBA). It also refers to the findings of a small-scale usability study and the cognitive 
laboratory research conducted for the 2012 NAEP grade 4 pilot WCBA. In addition, this report makes 
occasional references to the 2010 NAEP grade 4 paper-based pilot writing assessment and to the 
published 2011 NAEP grade 8 and 12 computer-based writing assessment reports 
( http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2012470 ). Because the focus of the report is the 
2012 grade 4 pilot WCBA, this methods section provides a brief description of the 2012 pilot WCBA data 
collection and the usability and cognitive laboratory studies and explains the statistical method used in 
this report. Additional details about NAEP writing assessment (e.g., framework, question format, 
sample of grade 8 and grade 12 WCBAs) may be found on the NAEP website 
( http://nces.ed.gov/nationsreportcard/writing/ ). 

Participants 

2012 pilot WCBA The 2012 grade 4 writing pilot assessment was administered to 10,400 students 
selected to be nationally representative of all grade 4 students in the United States. High-minority public 
schools (i.e., public schools with over 15 percent Black and Hispanic combined enrollment) were 
oversampled to ensure reliable estimates of these minority groups. Among the participants, 57 percent 
were White, 16 percent were Black, 18 percent were Hispanic, 5 percent were Asian, and 4 percent 
represented other racial/ethnic groups. About 50 percent were male students, 91 percent were non- 
English language learners (ELLs), and 89 percent did not have any disabilities. 

NAEP typically reports only operational assessment results, therefore, there is not an established 
process to generate the sampling weights that are needed to calculate weighted school or student 
response rate for the pilot assessment. However, the preliminary sampling weights which are created 
to adjust for unequal probability of selection are available at the school level, but not at the student 
level. Therefore, as a rough estimate, the school response rate was calculated using the preliminary 
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sampling weights. The overall weighted school response rate was about 87 percent. Detailed school 
response rates by census region can be found in Appendix B. For individual students, the preliminary 
weight is not in a format that readily yields the calculation of weighted student response rate because 
the sampling rate within some schools were not available when the preliminary weight were produced. 
Therefore, the unweighted student response rate was calculated. The overall unweighted student 
response rate was about 95 percent. Detailed student response rates by census region can also be 
found in Appendix B. Please note that the unweighted student response rates are high and uniform 
across census regions - all around 95 percent. It is highly likely that on a weighted basis the response 
rates will be very little different and the overall rate will almost certainly round to something in the 
range of 94-96 percent. To serve as references, the overall student response rate for the 2012 
operational assessment was about 95 percent. In addition, the 2013 grade 4 Reading operational 
assessment showed a similar overall level of response rate and very little variation in response rates by 
race/ethnicity, gender, ELL status, disability status and national school lunch program (NSLP) eligibility. 

Usability study: In March and April of 2011, a usability study was conducted by Fulcrum IT Services to 
test the design and function of the computer system to be used in the writing pilot assessment (Fulcrum 
IT 2011). As part of the study, the participating students took a 5-minute test to assess their typing 
performance. In total, 60 fourth-grade students were recruited from one rural school, two suburban 
schools, and two urban schools in West Virginia, Virginia, and Florida. The participating schools were 
each asked to select six boys and six girls who would (1) constitute a representative sample of the school 
population and (2) have a range of computer experience and abilities. The students in the usability study 
had diverse ethnic backgrounds and were generally from low- to middle-income families. 

Cognitive laboratory research: In addition to the usability study, cognitive laboratory research was 
conducted in 2011 to explore fourth-grade students' reactions to computer-based writing tasks and 
accompanying stimuli. Fifty students (23 males and 27 females) of varied race/ethnicity participated in 
the study. Students had up to 30 minutes to write a response to a task (out of 11 tasks in total). The 
sessions were observed by a facilitator and by NCES staff. 

Other data source In addition to the data sources listed above, this report also refers to the findings 
from the 2011 NAEP grade 8 and 12 computer-based writing assessments. 

Writing pilot assessment approach 

In the 2012 grade 4 pilot WCBA, sampled students were randomly assigned 2 out of 36 writing tasks and 
given 30 minutes to complete each task using laptops loaded with basic word processing software. 

These tasks were presented to students in different ways— using text, audio, video, and photographs— 
and were designed to measure one of three communicative purposes: 

• to persuade, in order to change the reader's point of view; 

• to explain, in order to expand the reader's understanding; and 

• to convey experience (real or imagined), in order to share experiences with the reader. 
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Before being presented with the tasks, students were given a tutorial to familiarize them with the way 
material is presented on the computer screen and the NAEP custom-developed interface system. The 
tutorial did not show students how to use the common editing tools, such as cut or paste. Students 
completed their writing tasks on laptop computers provided by NAEP. 

Student responses were scored on a 6-point rating scale— from a lowest score of 1 to a highest score of 
6— across the three writing demands— language facility, development of ideas, and organization of 
ideas. The grade-specific scoring was based on a holistic approach— i.e., it considered the written 
response in its entirety rather than its three components separately. 

Analysis 

Because of the pilot nature of the 2012 NAEP grade 4 WCBA, scale scores and achievement-level results 
traditionally reported in the NAEP report card for operational assessments were not produced. Instead, 
the average score of student responses to two assigned tasks was calculated to indicate student writing 
performance. Average score was considered as a measure of writing performance even though two 
tasks for each student have two different purposes. This approach is reasonable, considering that NAEP 
writing scale is one unidimensional scale with no subscales for separate purposes. Three performance 
levels (low, middle and high) were used in the analysis. Those three performance levels do not have any 
direct correspondence with the NAEP achievement levels (Basic, Proficient, and Advanced). However, 
the lowest performing level (i.e., the lowest performing 20 percent) might be considered to correspond 
approximately to the below Basic level, since the grade 8 and 12 data collected in fully operational 
assessments in 2011 indicate that students below Basic level generally correspond to the bottom 20 
percent of students. 6 

All analyses in this report were based on a sample which was weighted to adjust for unequal probability 
of selection 7 . However, as a pilot assessment, the sample was not weighted to adjust for school and 
student nonresponses. Because the sample was not adjusted to account for potential nonresponse bias, 
accurate standard errors could not be estimated, so they were not calculated for the average scores and 
percentage estimates presented here. As a result, statistical significance tests were not conducted. In 
lieu of statistical significance tests, the findings discussed in this report represent "substantive" results, 
which are defined as 7 percentage points or higher for percentage differences based on Rosenthal's 
(1996) study; 8 effect sizes greater than .20 for score or other numerical differences; and correlation 


6 The most recent full-scale grade 4 writing assessment was administered in 2002. In that year, 14 percent of grade 
4 students and 15 percent of grade 8 students performed at the below Basic level. 

7 Note that in some cases, the preliminary weight was calculated based on an educated guess as to what the within 
school sampling rate would actually be because the student data were not available when the preliminary weights 
were produced. 

8 A 7-percentage-point difference was applied for percentages ranging from 15 to 85. However, when either of the 
percentages to be compared was out of this range, a 5-percentage-point difference was used as the criterion for a 
substantive difference (Rosenthal 1996). 
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coefficients equal to or greater than .10. The criteria for numerical differences and correlation 
coefficients are based on Guideline 5-1-4H in the NCES statistical standards (U.S. Department of 
Education 2012). For some data tables, even though the overall relationship between two variables was 
not substantive, particular categories are discussed if percentage or other numerical differences are 
substantive. In this report, scores or word counts comparisons are always based on the average values 
of two groups. To avoid wordy statements, the term "on average" is not used all the time. All 
nonsubstantive findings are presented in appendixes (Appendix D, E, and F). 

Only demographic groups with pilot sample sizes large enough to report for most of the results (i.e., 
White, Black, Hispanic and Asian) were included in the analysis. 

In this report, all results related to writing performance are based on eligible responses only. Word 
counts related tables without performance data include ineligible responses such as blank or off-task. 

9. Cautions in interpretation 

There are a few limitations in interpreting the results presented in this report related to the sample, 
teacher survey questions, comparison inferences, and causal relationship inferences. 

Sample 

The first limitation is about the generalizability of the results. The statistics presented in this report are 
estimates based on a nationally representative sample of 10,400 grade 4 students. While the sample 
was weighted to adjust for the unequal probability of selection, as a pilot assessment, it was not 
adjusted to account for nonresponse bias. As a result, the weighted sample may not completely 
represent the fourth-grade population (especially at the subgroup level, such as race/ethnicity) in the 
United States. Readers are cautioned about generalizing the findings presented here to the larger 
population of grade 4 students. 

Teacher survey questions 

The second limitation is related to the teacher survey questionnaire. During the 2012 fourth-grade 
writing pilot assessment, teachers of the participating students completed a teacher questionnaire. In 
the questionnaire, teachers were asked six questions related to computer instruction and use. These 
questions asked what level of keyboarding skills teachers expected for their students based on their 
keyboarding instruction, how often students were asked to use the Internet to find information to 
include in their writing, and about the availability of the Internet for classroom instruction. Flowever, 
since the teachers were not linked to the participating students, 9 it was not possible to analyze how 
their questionnaire responses relate to the student performance data. In the absence of such 


9 The current practice in NAEP is to link teachers to the participating students for operational assessments only. 


Performance of fourth-grade students in the 2012 NAEP computer-based writing pilot assessment 10 


Chapter 1. Introduction 


information from teachers, it is not possible to fully delineate students' prior exposure to instruction in 
keyboarding or the use of editing tools. Nor is it possible to know if language arts teachers asked their 
fourth-grade students to use a computer for editing or to write drafts for final versions of their 
texts. Consequently, it is not possible to use teacher's responses to examine whether instruction in 
keyboarding or the use of editing tools is associated with student performance. 

Interpretations or inferences based on comparisons between paper- and 
computer-based assessment results 

The third limitation concerns making interpretations or inferences based on the comparisons between 
the 2010 paper- and the 2012 computer-based assessment results presented in chapter 2 of this report. 
There are several differences between the two assessments, which were not possible to be taken into 
account in making comparisons. For example, first, there could be a potential cohort difference between 
2010 and 2012. It is conceivable that the fourth-grade population in 2010 was different in some ways 
from the fourth-grade population in 2012. Second, the scoring guides were somewhat simplified for the 
computer-based assessment in 2012 as compared to the paper-based assessment (See Text Length 
section in Chapter 2 for details). Because the potential effects of the differences between the two 
assessment years are unknown, readers are cautioned when making any interpretations or inferences 
based on performance comparison data across the two assessment years. 

Relationship between variables 

Finally, NAEP is not a randomized experimental study. The design of the NAEP data collection process 
does not allow for drawing inferences from the data about causal relationships. In particular, it is 
difficult to determine the direction of the relationship. For example, do differences in editing tools cause 
differences in text length or do text length differences cause differences in editing tool use? Moreover, 
there may be plausible alternative factors that could explain any relationship found between the use of 
editing tools and text length. For example, due to teacher expectations, low-performing students may 
be asked less frequently than other students to write on the computer for schoolwork or 
homework. Conversely, low-performing students may be given more frequent computer-based writing 
assignments as a compensatory strategy— resulting in a seemingly counterintuitive relation between 
performance and computer-based writing assignments. 
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This chapter presents findings relevant to the central question this study sought to answer— i.e., 
whether or not fourth-graders could fully demonstrate their writing abilities on the computer on the 
2012 NAEP computer-based pilot writing assessment. The findings focus on fourth-graders' overall 
performance, their performance on tasks common to the 2012 NAEP computer-based and 2010 NAEP 
paper-based assessments, the length of their responses, and, finally, their typing speed. 

1. Overall performance 

To answer the central research question— Can fourth-graders fully demonstrate their writing abilities on 
the computer?— fourth-graders' overall performance was first examined. 

Fourth-graders' overall writing performance on the 2010 paper-based assessment and the 2012 
computer-based assessment was similar. The average p-value 10 for their responses to the writing tasks 
was .37 for writing on paper and .40 for writing on the computer. It is conceivable that fourth-graders' 
overall writing performance on the computer would have been lower than their performance on paper 
had the scoring guides not been made somewhat more lenient in the computer-based assessment (see 
Cautions in Interpretation). 

Figure 2 shows the percentage distribution of student responses by score category. Each category for 
each task has some responses and thus did not have to be collapsed with any other category; that is, the 
scorers were able to apply the full 6-level scoring rubric to all tasks. In this sense, the 2012 fourth-grade 
computer-based assessment administration was successful. 


10 The percentage correct for the writing task was calculated as the sum of the percentage of students in each score category 
weighted by the magnitude of each score category and then divided by the maximum score. For example, in the case of the 
2012 pilot assessment, the percentage correct of 40 percent was calculated as follows: [0 (point)*14(percentage) + 1 
(point)*26(percentage) + 2 (point)*28(percentage) + 3 (point)*19(percentage) + 4 (point)*10(percentage) + 5 
(point)*4(percentage)]/ 5 (maximum score) = 40. The p-value is the percentage correct divided by 100 to put it on the 0 to 1 
scale rather than the 0 to 100 (percent) scale. 
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Figure 2. Percentage distribution of responses in the NAEP computer-based writing pilot assessment at 
grade 4, by score category: 2012 


Percentage of responses 



NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Grade 4 Writing Pilot Assessment. 

However, figure 2 also shows that more than two-thirds of fourth-graders' responses (68 percent) 
received scores in the bottom half of the 6-point rating scale (1, 2, or 3), which results in the average p- 
value of .40. Across grades, a relatively higher percentage of the responses of fourth-grade students in 
the 2012 computer-based pilot assessment than eighth- or twelfth-grade students in the 2011 
computer-based operational assessments scored at the low end of the scale (at 1 and 2). Specifically: 

• 40 percent of fourth-graders' responses received scores of 1 or 2. 

• 20 percent of eighth-graders' responses received scores of 1 or 2. 

• 15 percent of twelfth-graders' responses received scores of 1 or 2. 
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2. Performance on tasks common to the paper- and computer-based 
assessments 

To examine mode effect as a potential explanation of fourth-graders' difficulty writing on the computer, 
their performance on 15 tasks common to the paper- and computer-based NAEP writing assessments 
was compared. 11 Of the 15 common tasks, 9 tasks had minor to no revisions, 5 tasks were 
reconceptualized to include video stimuli, and 1 task was simply reclassified from Explain to Persuade. 

When only the 15 common tasks were considered, the average p-values appeared to be even more 
similar (.40 for writing on paper and .41 for writing on the computer) than the average p-values for all 36 
tasks (.37 and .40, respectively). 

Restricting the dataset to only the common tasks (and thereby minimizing the effects of confounding 
factors, such as task difficulty and accessibility) led to different results. The proportion of responses in 
the top two levels of the 6-point scoring scale was substantially higher than on the computer compared 
to on the paper (15 percent vs. 10 percent), whereas the proportion of responses in the bottom two 
levels was not different (figure 3). Note that the comparison is not at individual score category, but 
rather at the three performance levels. 


11 The comparison did not take into account a potential cohort difference between 2010 and 2012 or other 
unknown differences such as motivation. It is conceivable that the fourth-grade population in 2010 was different in 
some ways from the fourth-grade population in 2012. Similarly, the comparison did not take into consideration a 
potential impact of simplifying the 2010 scoring rubrics for the 2012 computer-based assessment. 
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Figure 3. Percentage distribution of responses based on 15 tasks common to the 2010 NAEP paper-based 
pilot writing assessment and the 2012 NAEP computer-based pilot writing assessment at grade 
4, by score category 


Percentage of responses 



■ Paper-based 

■ Computer-based 


Score category 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2010 and 2012 Grade 4 Writing Pilot Assessments. 


Moreover, when fourth-graders' performance on paper was compared with that on the computer using 
average scores for low, middle, and high performers, the data supported the previously reported finding 
using the 6-point scoring scale. Specifically, the data revealed that high performers— i.e., those in the 
top 20 percent— scored substantially higher on the computer than on the paper assessment (effect size 
= 0.56). Low-performing students— representing the bottom 20 percent— as well as middle-performing 
students (representing the middle 60 percent) did not appear to benefit from using the computer, with 
effect sizes of 0.05 and 0.16, respectively (table 2 - 1 ). 


Table 2-1. Fourth-graders' performance comparison between writing on paper and writing on the 
computer, by performance level: 2010 and 2012 



Paper 

Computer 

Effect size 

All students 

2.98 

3.08 

0.08 

Low 

1.50 

1.47 

0.05 

Middle 

2.87 

2.99 

0.16 

High 

4.36 

4.82 

0.56 


NOTE: The low, middle, and high performance levels represent the bottom 20 percent, middle 60 percent, and top 
20 percent of students in the performance, respectively. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2010 and 2012 Grade 4 Writing Pilot Assessments. 
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These findings suggest potential differential effects of the computer on the writing performance of the 
high and low performing fourth-graders, thus widening the achievement gap. The term "potential" is 
used here because it is unknown whether (or to what extent) the somewhat more lenient scoring guides 
in the 2012 computer-based assessment (described under "Text Length" below) might have impacted 
the scores. 

Assuming the unknown potential effects of the more lenient scoring rubrics are constant across 
performance levels (because the scoring guides were simplified across all 6 score points) the use of the 
computer is likely to be the key factor associated with the differential change in the performance of 
high- and low-performing fourth-graders on the computer versus paper. 

The likely scenario is that the computer had positive effects on the writing performance of the high- 
performing group, but no or negative effects on the performance of other groups. Although there is no 
absolute evidence for the positive effect of the computer for high performers given the fact that the 
assessment was not designed to examine mode effect, this interpretation builds on the following 
empirical evidence. First, if the computer had similar effects on the three performance groups, there 
should have been changes of similar magnitude for students at all three performance levels. However, 
the high-performing students scored substantively higher on the computer than on paper, while the 
lower performing groups did not. Second, there are contextual data indicating that the higher- 
performing students had greater prior exposure to the computer and lower performing students had a 
greater preference for paper than higher performing students. Finally the findings are consistent with a 
meta-analysis of studies published between 1992 and 2002 by Goldberg, Russell, and Cook (2003), who 
found some evidence of the positive overall effects of word processing on the quantity and quality of 
student writing at the middle and high school and less so at the elementary school levels. 

An alternative scenario is that the computer had negative effects for all, with the least negative effect 
on high-performing students. Considering the constant positive effect of the scoring rubrics, this 
alternative scenario suggests that the negative effects of the computer is not as large as the positive 
effects of scoring guides for the high-performing groups, and that this could explain why they obtained a 
higher average score on the computer. For the low performing group, the positive effects of the scoring 
rubrics and the negative effects of the computer might have been cancelled out, and that this could 
explain why they did not show substantive changes between the two assessment years. This scenario of 
the negative computer effect, in varying degrees, on the writing of all students including high 
performers is unlikely, considering the NAEP contextual data and the literature discussed above. 

3. Text length 

To support the results showing that low- and middle-performing students may not be able to exploit the 
computer to improve their writing performance on the computer, as do high-performing students 
appear to do, response lengths in the paper- and computer-based assessments were compared. (The 
word count was conducted by computer for the computer-based assessments and by manual counting 
of a random selection of responses for the paper-based assessments.) 
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As shown by figure 4, the average number of words produced by fourth-graders was smaller on the 
computer-based pilot assessment than on the paper-based pilot assessment (110 vs. 159). In contrast, 
the average number of words for eighth- and twelfth-graders was larger on the computer-based 
assessment than on the paper-based assessment (300 vs. 155 for grade 8; 389 vs. 235 for grade 12). 

Figure 4. Word count per task, by mode of writing assessment and grade: Various years, 2007-2012 


Grade level 



0 100 200 300 400 500 


■ Paper-based 

■ Computer-based 


Word count per task 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2010 and 2012 Grade 4 Writing Pilot Assessments, 2007 and 2011 Grade 8 and 12 Writing 
Assessments. 


In the computer-based assessment, it was also possible to examine the relationship between text length 
and students' performance level. Low-performing fourth-graders produced fewer words (60) than did 
middle- and high-performing fourth-graders (104 and 179, respectively) (table 3-2). (Due to a lack of 
data, it was not possible to examine the relationship between text length and students' performance 
level or scores in the paper-based assessment. It might be reasonable, however, to assume that the 
overall relationship between text length and writing score observed in the computer-based pilot 
assessment would hold true for the paper-based pilot assessment as well.) 

What explains the findings that fourth-graders' overall writing performance on paper and computer is 
similar, yet they produce substantially fewer words on the computer (110) than on paper (159), even 
though text length is positively correlated with performance on the computer? 

Changes in the scoring guide for the computer-based assessment may account for the fact that fourth- 
graders' overall average scores were similar between paper and computer when average responses 
were much longer on paper. The scoring guides for grade 4 were simplified for the computer-based 
assessment because the overall average score of the 2010 paper-based pilot assessment was relatively 
low. These changes were applied across all 6 score levels and across all purposes of writing. In terms of 
Language Facility, the requirements for sentence structure and word choice were toned down. In terms 
of the Development of Ideas, "explains in an insightful way..." was changed to simply "explains..." and 
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"provides details that..." was changed to "across the response, there is evidence of being able to ..." In 
terms of Organization of Ideas, several changes were made (e.g., "logical progression of ideas" was 
changed to "Ideas progress clearly"). In other words, fourth-graders' overall writing performance might 
have been lower on the computer-based assessment, had the scoring rubrics not been simplified. 

4. Typing rate 

Considering that students' typing speed is relevant to the amount of text produced, the typing rate of 
fourth-graders was examined next. Typing rates were assessed in a usability study conducted in 
preparation for the 2012 assessment (Fulcrum IT 2011). In that study, a typing test was given to 60 
fourth-graders (30 boys and 30 girls) on a laptop to determine their typing speed on a passage typed for 
5 minutes. The data indicated that 29 percent of the sampled fourth-grade students typed fewer than 
10 words per minute. Fourth-graders' "average" typing speed was 12 words per minute, compared to 30 
words per minute for eighth-graders (figure 5). 12 Fourth-graders' "most common" typing speed was 12 
words per minute, compared to 22 words per minute for eighth-graders. 13 

Figure 5. Typing rate comparison between grade 4 students and grade 8 students, by central tendency 
measure of typing speed: 2011 



Average typing speed Most common typing speed 

NOTE: Typing speed was determined based on a 5-minute typing test. 

SOURCE: Fulcrum IT. (2011, May). National Assessment of Educational Progress— Writing Computer-Based 
Assessment Grade 4 Small Scale Usability Study 


"Average typing speed" refers to the average number of words typed per minute by the students participating in 
the usability study. 

13 "Most common typing speed" is the average number of words typed per minute by most of the students 
participating in the usability study. 
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Figure 6 shows that there were some fourth-grade students who typed fewer than five words per 
minute. 

Figure 6. Typing rate of grade 4 students, by descriptive statistics: 2011 
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speed 


NOTE: The typing speed was determined based on a 5-minute typing test. 

SOURCE: Fulcrum IT. (2011, May). National Assessment of Educational Progress— Writing Computer-Based 
Assessment Grade 4 Small Scale Usability Study 


In summary, while the average p-values for writing on paper and the computer were similar, low- and 
middle-performing fourth-graders did not appear to benefit from using the computer, while high- 
performing students appear to have been able to exploit the computer to improve their writing, thus 
further widening the achievement gap. Overall, fourth-graders produced shorter texts when writing on 
the computer (110) than when writing on paper (159), and they demonstrated limited typing skills 
(average speed of 12 words per minute) compared to eighth-graders (average speed of 30 words per 
minute). 
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Chapter 3. Relationships Among Text Length, the Use 
of Editing Tools, and Writing Performance 

This chapter examines how fourth-graders' writing performance on the 2012 NAEP computer-based 
pilot writing assessment was related both to the length of their responses as well as to the editing tools 
they used to compose, revise, and organize their responses. It was hypothesized that (1) text length is 
positively related to writing performance; (2) the use of editing tools is positively related to writing 
performance; and (3) text length is positively related to the use of editing tools. These relationships are 
illustrated in figure 7 by the blue arrows and boxes. 

Figure 7. Conceptual model of the relationship among fourth-graders' writing performance, text length, 
and the use of editing tools 
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1. The relationship between text length and writing performance 

The relationship between text length and writing performance was examined in three different ways: 
the relationship between word count and raw writing score; the relationship between word count and 
performance level; and the relationship between the number of key presses and performance level. As 
hypothesized, all three relationships were positive. 

Average word count by raw writing score 

Table 3-1 shows the average, minimum and maximum word counts by raw writing score. As 
hypothesized, the raw writing score was positively related with the word count. The relationship 
between raw score and word count was relatively high (r = .61). 

Table 3-1. Average, minimum, and maximum word count of fourth-grade students' responses, by raw 
writing score: 2012 


Raw score 


Word count 


Average 

Minimum 

Maximum 

1 

56 

1 

588 

2 

80 

11 

629 

3 

110 

13 

881 

4 

136 

37 

639 

5 

182 

59 

614 

6 

233 

67 

768 


NOTE: Boxplot for the distribution of word count by raw score is presented in Appendix F (figure F-l) 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


Average word count by performance level 

As shown in Table 3-2, low-performing students produced fewer words (60) than middle- and high- 
performing students (104 and 179, respectively). The difference was larger between middle- and high- 
performing students (75 words) than between low- and middle-performing students (44 words). 

Table 3-2. Average word count per task of fourth-grade students, by performance level: 2012 




Performance level 




Low 

Middle 

High 

All students 

Average word count per task 

60 

104 

179 

110 


NOTE: The low, middle, and high performance levels represent the bottom 20 percent, middle 60 percent, and top 
20 percent of student performance, respectively. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 
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Key presses and performance level 

The number of key presses can be considered an indirect measure of text length. Low-performing 
fourth-grade students had fewer key presses than higher performing students, which is similar to the 
results for word counts. For example, 9 percent of low-performing fourth-graders had over 1,000 key 
presses, compared to 25 percent of middle-performing and 71 percent of high-performing fourth- 
graders (table 3-3). 

Table 3-3. Percentage distribution of fourth-grade students, by performance level and number of key 
presses: 2012 


Number of 

Performance level 


All students 

key presses 

Low 

Middle 

High 

0-1,000 

91 

76 

29 

69 

1,001-2,000 

9 

24 

63 

29 

2,001-3,000 

# 

1 

8 

2 

> 3,001 

ft 

ft 

1 

ft 


# Rounds to zero. 

NOTE: Detail may not sum to totals because of rounding. The low, middle, and high performance levels represent 
the bottom 20 percent, middle 60 percent, and top 20 percent of student performance, respectively. The term key 
press represents each keystroke that students make in the process of completing their response to the writing 
prompt, including letters, the spacebar, delete, and backspace. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

The relationship between performance level and the number of key presses was moderate (r = .43), 
supporting the finding of a positive relationship between writing performance and text length. 

2 . The relationship between the use of editing tools and writing 
performance 

The relationship between the use of editing tools and writing performance was examined separately for 
each of the 12 editing tools. Three of the tools— spellcheck, accepting spelling corrections, and 
backspace— were positively correlated with writing performance (r = .15 for spellcheck and writing 
performance level; r = .20 for accepted automated spelling corrections and writing performance level, r 
= .18 for backspace and writing performance level); one tool— cut— was negatively correlated to writing 
performance (r = -.17). 
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Language-related tools 

Spellcheck and accepting spelling corrections 

As shown in table 3-4: 

• A higher percentage of fourth-grade students at the low performance level (35 percent) than at 
the middle (24 percent) and high (17 percent) performance levels did not use spellcheck. 

• The percentage of fourth-grade students who used spellcheck 4 or more times— via any of the 
three available methods (see Section 5. "Defining the Editing Tools" in chapter 1)— was lower at 
the low performance level (30 percent) than at the middle and high performance levels (40 and 
47 percent, respectively). 

• A higher percentage of fourth-grade students at the low performance level (61 percent) than at 
the middle and high performance levels (46 and 32 percent, respectively) did not accept any 
automated spelling corrections. 

• Conversely, the percentage of fourth-grade students at the low performance level who accepted 
at least 4 automated spelling corrections (18 percent) was lower than for those at the middle 
and high performance levels (33 percent and 45 percent, respectively). 

Table 3-4. Percentage distribution of fourth-grade students who used spellcheck editing tools, by 
performance level and number of uses: 2012 


Number of uses of spellcheck editing 

Performance level 



tools 

Low 

Middle 

High 

All students 

Spellcheck 

0 

35 

24 

17 

25 

1 

16 

15 

12 

14 

2 

11 

11 

12 

11 

3 

8 

10 

12 

10 

>4 

30 

40 

47 

39 

Accepted automated spelling 
corrections 

0 

61 

46 

32 

46 

1 

10 

9 

8 

9 

2 

6 

6 

8 

6 

3 

5 

6 

7 

6 

>4 

18 

33 

45 

32 


NOTE: Detail may not sum to totals because of rounding. The low, middle, and high performance levels represent the 
bottom 20 percent, middle 60 percent, and top 20 percent of student performance, respectively. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 
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Thesaurus and thesaurus replacement 

Students also had the option of using the thesaurus and thesaurus replacement tools to enhance their 
writing. Contrary to the hypothesis, writing performance was not related to the use of these tools. A 
table showing the percentage of students who used the thesaurus editing tools by writing performance 
level is available in appendix C (table C-l). 

Emphasis-related tools 

Bold, italic, and underline 

As with the thesaurus tools, there were few differences in the use of the emphasis-related editing tools 
by performance level. A table showing the percentage of students who used the emphasis-related 
editing tools by writing performance level is available in appendix C (table C-2). 

Revision-related tools 

Cut, copy, paste, delete, and backspace 

Most fourth-grade students (80 percent or above) did not use any of the revision-related editing tools 
except for the backspace key, which almost all students used at least once. Out of the five revision- 
related editing tools, two— backspace and cut— had a substantive correlation with writing performance, 
one of which was positive (for backspace, r = .18) and one of which was negative (for cut, r = -.17). Highl- 
and low- performing students differed in the frequency with which they used the copy tool, but the 
correlation between the frequency of use of the copy tool and student performance was not 
substantive. Specifically: 

• The use of the backspace key differed by performance level. For example, the percentage of 
fourth-grade students who used the backspace key 100 times or fewer was higher for those at 
the low performance level (64 percent) than for those at the middle or high performance levels 
(57 percent and 38 percent, respectively) (table 3-5). 

• The percentage of fourth-grade students who used the cut tool at least once was higher for 
those at the low performance level (33 percent) than for those at the middle and high 
performance levels (18 percent and 11 percent, respectively). 

• The percentage of fourth-grade students who used the copy tool at least once was higher for 
those at the low performance level (19 percent) than for those at the high performance level (10 
percent). 
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Table 3-5. Percentage distribution of fourth-grade students who used the backspace, cut, and copy tools, 
by performance level and number of uses: 2012 


Number of 


Performance level 



uses of revision 
editing tool 

Low 

Middle 

High 

All students 

Backspace 

0 

1 

# 

# 

# 

1-100 

64 

57 

38 

55 

101-200 

26 

32 

43 

33 

>201 

9 

10 

20 

12 

Cut 

0 

67 

82 

89 

80 

> 1 

33 

18 

11 

20 

Copy 

0 

81 

87 

90 

86 

> 1 

19 

13 

10 

14 


# Rounds to zero. 

NOTE: Detail may not sum to totals because of rounding. The low, middle, and high performance levels represent 
the bottom 20 percent, middle 60 percent, and top 20 percent of student performance, respectively. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

The use of two of the revision-related editing tools— delete and paste— was not related to writing 
performance. A table showing the percentage of students who used the two revision-related editing 
tools by writing performance level is available in appendix C (table C-3). 

3. The relationship between text length and the use of editing tools 

It was hypothesized that text length is positively related to frequency of the use of editing tools. The 
relationship between text length and the use of editing tools was examined separately for each of the 12 
editing tools. The use of two tools— accepting spelling corrections and the backspace key— was found to 
be positively correlated with text length (r = .17 and .34, respectively); the use of one tool— cut— was 
found to be negatively correlated (r = -.18). 

Language-related 

Spellcheck and accepting spelling corrections 

As shown in table 3-6: 

• The average word count was similar among fourth-grade students who accepted 0 to 3 
automated spelling corrections (ranging from 99 to 106 words). Students who accepted 4 or 
more automated spelling corrections produced 21 to 28 more words (about 127). 

• In general, fourth-grade students who accepted automated spelling corrections more frequently 
had a higher word count. 


Performance of fourth-grade students in the 2012 NAEP computer-based writing pilot assessment 25 


Chapter 3. Relationships Among Text Length, the Use of Editing Tools, and Writing Performance 


Table 3-6. Average, minimum, and maximum word counts of fourth-grade students' responses, by 
number of automated spelling corrections accepted: 2012 


Number of automated spelling Word count 


corrections accepted 

Average 

Minimum 

Maximum 

0 

102 

0 

586 

1 

99 

0 

438 

2 

105 

3 

652 

3 

106 

3 

573 

>4 

127 

0 

594 


NOTE: Boxplot for the distribution of word count by the number of spelling corrections accepted is presented in 
Appendix F (figure F-2) 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

There were no substantive differences in word count by the frequency of use of spellcheck. A table 
showing the word count by the use of spellcheck is available in appendix C (table C-4). 

The thesaurus 

The use of the two thesaurus-related editing tools did not have any substantive relationship with text 
length. A table showing the word count by the frequency of use of each thesaurus-related editing tool is 
available in appendix C (table C-5). 

Emphasis-related 

The use of the three emphasis-related editing tools did not have any substantive relationship with text 
length. A table showing the word count by the frequency of use of each emphasis-related editing tool is 
available in appendix C (table C-6). 

Revision-related 

Only two of the five revision-related editing tools— the backspace key and cut tool— had a substantive 
relationship with text length. Specifically: 

• Fourth-grade students who used the backspace key more frequently had a higher word count 
than those who used it less frequently (r = .34). For example, students who used the backspace 
key more than 200 times composed responses of about 154 words, while students who did not 
use the backspace key at all composed responses of about 51 words (table 3-7). 

• Fourth-grade students who did not use the cut tool at all wrote responses that were about 27 
words longer than those who used it at least once (r = -.18). 
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Table 3-7. Average, minimum, and maximum word counts of fourth-grade students' responses, by 
number of uses of backspace and cut tools: 2012 




Word count 



Number of uses of revision editing tool 

Average 

Minimum 

Maximum 

Backspace 

0 

51 


0 

185 

1-100 

90 


0 

500 

101-200 

128 


0 

586 

>201 

154 


0 

652 

Cut 

0 

116 


0 

652 

> 1 

89 


0 

500 


NOTE: Boxplot for the distribution of word count by the number of backspace and cut is presented in Appendix F 
(figure F-3a and figure F-3b, respectively). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

There were no substantive differences in word count by the use of the delete, copy, or paste tools. A 
table showing the word count by the frequency of students' use of each these tools is available in 
appendix F (table F-7). 


Performance of fourth-grade students in the 2012 NAEP computer-based writing pilot assessment 27 


Chapter 4. Relationships of Prior Exposure to Writing 
on the Computer With Text Length, the Use of Editing 
Tools, and Writing Performance 

This chapter examines a set of contextual variables relevant to fourth-graders' prior exposure to writing 
on the computer in relation to all of the variables examined in chapter 3: (1) the length of students' 
responses, (2) writing performance, and (3) use of editing tools on the 2012 NAEP computer-based pilot 
writing assessment. It was hypothesized that prior exposure to writing on the computer is positively 
related to all three factors. These relationships are illustrated in Figure 8 by the blue arrows and boxes. 

Figure 8. Conceptual model of the relationships of prior exposure to writing on the computer with text 
length, the use of editing tools, and writing performance 


Prior Exposure to Writing on Computer 



Writing Performance 


1. The relationship between text length and prior exposure to writing 
on the computer 

It was hypothesized that prior exposure to writing on the computer is positively related to text length. 
The relationship between text length and prior exposure to writing on the computer was examined for 
seven related contextual variables. Three of these variables were found to be positively correlated to 
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text length: students' having access to the Internet at home, using the Internet to get information to 
include in their writing, and writing to friends and family using the Internet. 

Internet access at home 

Table 4-1 shows that fourth-grade students with Internet access at home composed longer responses 
(118 words, on average) than students who had no Internet access at home (83 words, on average). 
Internet access at home was positively correlated with the number of words produced (r = .25). 

Table 4-1. Average, minimum, and maximum word counts of fourth-grade students' responses, by access 
to the Internet at home: 2012 




Word count 


Access to Internet at home 

Average 

Minimum 

Maximum 

No 

83 

0 

439 

Yes 

118 

0 

652 


NOTE: Boxplot for the distribution of word count by access to the Internet at home is presented in Appendix F 
(figure F-4). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

Using the Internet to get information to include in their writing 

As expected, students who used the Internet more often to look for information to include in their 
writing wrote longer responses, on average, than students who did so less frequently (table 4-2). Thus, 
students who used the Internet to look for information to include in their writing once a week wrote 118 
words and students who did so 1-2 times a month or a few times a year wrote 116 words, whereas 
students who reported never or hardly ever having used the Internet to look for information to include 
in their writing wrote 102 words. The frequency of looking for information on the Internet for writing 
had a positive correlation with word count (r = .10). 
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Table 4-2. Average, minimum, and maximum word counts of fourth-grade students' responses, by how 
often students looked for information on the Internet to include in their writing: 2012 


Flow often students looked for information on 
the Internet to include in their writing 

Average 

Word count 
Minimum 

Maximum 

Never or hardly ever 

102 

0 

475 

1-2 times a month or a few times a year 

116 

0 

652 

Once a week 

118 

1 

586 


NOTE: Boxplot for the distribution of word count by the frequency of the use of the Internet to look for information 
to include in their writing is presented in Appendix F (figure F-5). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

Using the Internet to write to friends or family 

In general, there was a positive relationship between the length of fourth-grade students' responses and 
the frequency with which they used the Internet to write to friends or family ( r = .18). Specifically, 
students who used the Internet to write to friends or family once a week produced the longest 
responses (128 words on average), followed by students who did 1-2 times a month or a few times a 
year (118 words on average) and then, by students who never or hardly ever used the Internet to write 
to friends or family (99 words on average). 

Table 4-3. Average, minimum, and maximum word counts of fourth-grade students' responses, by how 
often students wrote to friends or family using the Internet: 2012 


Flow often students wrote to 
friends or family using the Internet 

Average 

Word count 
Minimum 

Maximum 

Never or hardly ever 

99 

0 

503 

1-2 times a month or a few times a year 

118 

0 

586 

Once a week 

128 

0 

652 


NOTE: Boxplot for the distribution of word count by the frequency of the use of the Internet to write to friends or 
family is presented in Appendix F (figure F-6). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

Four prior-exposure variables with no substantive relationship with text length 

Four contextual variables related to prior exposure to writing on the computer were found to have no 
substantive relationship with text length: whether or not students received instruction in keyboarding 
and word processing, how often students received computer-based writing assignments, how often 
students received computer-based writing tests with an extended constructed-response component, 
and how much time students spent each day writing on the computer for school assignments. 
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Tables for the average word count for each of these variables and a brief description of each table are 
available in appendix D as follows: 

• table D-l, for instruction in keyboarding and word processing; 

• table D-2, for computer-based writing assignments; 

• table D-3, for computer-based writing tests with an extended constructed-response component; 
and 

• table D-4, for writing on the computer for school assignments. 

2. The relationship between the use of editing tools and prior 
exposure to writing on the computer 

It was hypothesized that prior exposure to writing on the computer is positively related to writing 
performance. The relationship between editing tool use and the seven contextual variables related to 
prior exposure to writing on the computer was examined separately for each of the 12 editing tools. 

Two of the seven contextual variables— Internet access at home and writing to friends or family using 
the Internet— had a substantive relationship with one or two editing tools. 

Instruction in keyboarding and word processing 

The majority of fourth-grade students (88 percent) received instruction in keyboarding and word 
processing, based on school reports. However, instruction in keyboarding and word processing was not 
related to students' use of any of the 12 editing tools; the frequency of use of these tools was very 
similar for those who received instruction in keyboarding and word processing and those who did not. 

Tables for the percentage of students who used each editing tool by whether or not they received 
instruction in keyboarding and word processing are available in appendix D as follows: 

• table D-5-a and table D-5-b for the language-related editing tools; 

• table D-5-c for the emphasis-related editing tools; and 

• table D-5-d for the revision-related editing tools. 

Receiving computer-based writing assignments 

According to school reports, about 31 percent of fourth-grade students never or hardly ever received 
computer-based writing assignments in a school year, while 63 percent received such assignments a few 
times a year or once or twice a month, and 6 percent received the assignments at least once a week. 

The frequency of receiving computer-based writing assignments was not substantively correlated to 
students' use of any of the 12 editing tools. Frequency of using editing tools was very similar between 
those who received computer-based writing assignments in a school year and those who did not. 

There were, however, differences in the percentages of students who accepted automated spelling 
corrections and number of uses of the backspace key by the frequency with which they received 
computer-based writing assignments. 
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Table 4-4 shows that a higher percentage of students who never or hardly ever received computer- 
based writing assignments (51 percent) than those who received such assignments at least once a week 
(39 percent) did not accept automated spelling corrections. Conversely, a lower percentage of students 
who never or hardly ever received computer-based writing assignments (29 percent) than who received 
such assignments at least once a week (37 percent) accepted more than three automated spelling 
corrections. 

Table 4-4. Percentage distribution of fourth-grade students who accepted automated spelling 

corrections, by how often they received computer-based writing assignments and number of 
corrections accepted: 2012 


Number of 
automated spelling 
corrections 
accepted 

How often students received computer-based writing assignments 

All students 

Never or hardly ever 

1-2 times a month or 
a few times a year 

At least once a week 

All students 

31 

63 

6 

100 

0 

51 

45 

39 

46 

1 

9 

9 

10 

9 

2 

7 

6 

8 

6 

3 

5 

6 

6 

6 

>4 

29 

34 

37 

33 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


In addition, a lower percentage of fourth-grade students who received computer-based writing 
assignments at least once a week (48 percent) used the backspace key less frequently (i.e., 100 times or 
less) than those who never or hardly ever received computer-based writing assignments (56 percent) 
and those who received computer-based writing assignments 1-2 times a month or a few times a year 
(55 percent) (table 4-5). The correlation, however, was not substantive (r = -.03). 

Table 4-5. Percentage distribution of fourth-grade students who used the backspace key, by how often 
they received computer-based writing assignments and number of uses: 2012 



How often students received computer-based writing assignments 


All students 

Number of uses 
of backspace key 

Never or hardly ever 

1-2 times a month or 
a few times a year 

At least once a week 

All students 

31 

63 


6 

100 

0 

# 

# 


# 

# 

1-100 

56 

55 


48 

55 

101-200 

32 

33 


38 

33 

>201 

11 

12 


14 

12 


# Rounds to zero. 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 
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For the 10 editing tools that did not have any substantive relationship with how often fourth-grade 
students received computer-based writing assignments, tables for the percentage of students who used 
each editing tool by the frequency with which they received computer-based writing assignments are 
available in appendix D as follows: 

• table D-6-a and table D-6-b for the language-related editing tools; 

• table D-6-c for the emphasis-related editing tools; and 

• table D-6-d for the revision-related editing tools. 

Computer-based writing tests with an extended constructed-response 
component 

Most fourth-grade students (84 percent) never or hardly ever took computer-based writing tests with an 
extended constructed-response component, based on school reports. However, this contextual variable 
was not related to the use of any of the 12 editing tools; the frequency of use of these tools was very 
similar for those who took computer-based writing tests with an extended constructed-response 
component and those who did not. 

Tables showing the percentage of students who used each editing tool by the frequency with which they 
received computer-based writing tests with an extended constructed-response component are available 
in appendix D as follows: 

• table D-7-a and table D-7-b for the language-related editing tools; 

• table D-7-c for the emphasis-related editing tools; and 

• table D-7-d for the revision-related editing tools. 

Internet access at home 

Based on student reports, 77 percent of fourth-grade students had Internet access at home and 23 
percent did not (see table 4-6). Internet access at home was positively related to the two language- 
related editing tools: spellcheck and accepting automated spelling corrections. In addition, there were 
substantive differences— but not overall substantive correlations— for three other tools: thesaurus, 
bold, and Italics. 

As table 4-6 shows, students who said they did not have Internet access at home were more likely not to 
use the spellcheck tools than were those who said they had access. Specifically, 36 percent of students 
without Internet access at home did not use spellcheck, compared to 22 percent of students with 
Internet access; and 62 percent of students without Internet access at home did not accept automated 
spelling corrections, compared to 42 percent of students with access. In addition, students with Internet 
access at home were more likely to use spellcheck 4 or more times or accept automated spelling 
corrections 4 or more times (42 percent and 36 percent, respectively) than were students without 
access (30 percent and 20 percent, respectively). Both associations were substantive (r = .15 and .17, 
respectively). 
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Table 4-6. Percentage distribution of fourth-grade students with or without access to the Internet at 
home, by number of uses of spellcheck tools: 2012 


Number of uses of spellcheck tools 

Access to Internet at home 
No 

Yes 

All students 

All students 

23 

77 

100 

Spellcheck 

0 

36 

22 

25 

1 

17 

14 

14 

2 

9 

12 

11 

3 

8 

10 

10 

>4 

30 

42 

39 

Accepted automated spelling 
corrections 

0 

62 

42 

46 

1 

9 

9 

9 

2 

4 

7 

6 

3 

5 

6 

6 

>4 

20 

36 

32 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


As shown in Table 4-7, fourth-grade students who had no Internet access at home were more likely not 
to use the thesaurus than were students who had access (64 vs. 57 percent). In addition, students who 
had Internet access were more likely to use the thesaurus one or more times (43 percent) than were 
students who had no access (37 percent). 

Table 4-7. Percentage distribution of fourth-grade students with or without access to the Internet at 
home, by number of uses of thesaurus tool: 2012 


Number of uses of 

Access to the Internet at home 



thesaurus tool 

No 

Yes 

All students 

All students 

23 

77 

100 

Thesaurus 




0 

64 

57 

59 

1 

25 

30 

29 

>2 

12 

13 

13 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


Table 4-8 shows that students who had no Internet access at home were more likely not to use bold 
than were students who had access (67 vs. 59 percent) and were more likely not to use italics than were 
students who had access (74 vs. 68 percent). In addition, students who had Internet access at home 
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were more likely to use bold 3 or more times (16 percent) than were students who had no access (11 
percent). 

Table 4-8. Percentage of fourth-grade students with or without access to the Internet at home, by 
number of uses of bold and italics: 2012 


Number of uses of 

Access to the Internet at home 



bold and italics 

No 

Yes 

All students 

All students 

23 

77 

100 

Bold 

0 

67 

59 

60 

1 

15 

17 

17 

2 

7 

9 

8 

>3 

11 

16 

15 

Italics 

0 

74 

68 

69 

1 

13 

13 

13 

2 

4 

7 

6 

>3 

9 

12 

11 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


There were no substantive differences in the use of seven editing tools by Internet access at home: 
thesaurus replacement, underline, backspace, delete, cut, copy, and paste. Tables showing the 
percentage of students who used each editing tool by Internet access at home are available in appendix 
D as follows: 

• table D-8-a for the language-related editing tools; 

• table D-8-b for the emphasis-related editing tools; and 

• table D-8-c for the revision-related editing tools. 

Using the Internet to get information for writing 

Based on student reports, 32 percent of fourth-graders never or hardly ever looked for information on 
the Internet to include in their writing, 51 percent did so 1-2 times a month or a few times a year, and 
17 percent did so at least once a week (table 4-9). Although this contextual variable was not 
substantively correlated to any of the 12 editing tools, there were substantive differences in accepting 
automated spelling corrections and in the use of the backspace key between students who never or 
hardly ever looked for information on the Internet to include in their writing and those who did so more 
often. 

A higher percentage (50 percent) of students who never or hardly ever looked for information on the 
Internet to include in their writing did not accept any automated spelling corrections— and a lower 
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percentage (29 percent) accepted automated spelling corrections 4 or more times— than students who 
looked for information on the Internet 1-2 times a month or a few times a year (35 percent). 

Table 4-9. Percentage distribution of fourth-grade students who looked for information on the Internet 
to include in their writing, by how often they looked for information and number of 
automated spelling corrections accepted: 2012 


Number of automated 
spelling corrections 
accepted 

How often students looked for information on the Internet 

All students 

Never or hardly ever 

1-2 times a month or 
a few times a year 

At least once a 
week 

All students 

32 

51 

17 

100 

0 

50 

43 

45 

46 

1 

9 

9 

10 

9 

2 

6 

7 

6 

6 

3 

6 

6 

5 

6 

>4 

29 

35 

34 

33 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


In addition, a higher percentage (60 percent) of students who never or hardly ever looked for 
information on the Internet (table 4-10) to include in their writing used the backspace key less often 
(i.e., 1 to 100 times) than students who looked for information on the Internet to include in their writing 
at least once a week (50 percent) or 1-2 times a month or a few times a year (53 percent). 

Table 4-10. Percentage distribution of fourth-grade students who looked for information on the Internet 
to include in their writing, by how often they looked for information and number of uses of 
backspace key: 2012 


Number of uses of 
backspace key 

How often students looked for information on the Internet 

All students 

Never or hardly ever 

1-2 times a month or a 
few times a year 

At least once a 
week 

All students 

32 

51 

17 

100 

0 

# 

# 

# 

# 

1-100 

60 

53 

50 

54 

101-200 

30 

34 

37 

33 

>201 

10 

13 

13 

12 


# Rounds to zero. 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

There were no substantive differences in the use of 10 editing tools (spellcheck, thesaurus, thesaurus 
replacement, bold, Italics, underline, delete, cut, copy, and paste) by the frequency with which students 
used the Internet to look for information to include in their writing. Tables of the percentage of students 
who used each editing tool for this contextual variable are available in appendix D as follows: 
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• table D-9-al and table D-9-a2 for the language-related editing tools; 

• table D-9-b for the emphasis-related editing tools; and 

• table D-9-c for the revision-related editing tools. 

Using the Internet to write to friends or family 

According to the student survey data, 44 percent of fourth-graders said that they never or hardly ever 
wrote to friends or family using the Internet, 32 percent said they did so 1-2 times a month or a few 
times a year, and 24 percent said they did so at least once a week (see table 4-11). 

The frequency of writing to friends or family using the Internet was substantively correlated to the use 
of one of the 12 editing tools: the backspace key. While there were no overall substantive correlations, 
there were substantive differences in the use of one editing tool— accepting spelling corrections— 
between those who never or hardly ever wrote to friends or family using the Internet and those who did 
so at least once a week. The percentage of students who used the remaining 10 editing tools was very 
similar across all three frequencies of use (never or hardly ever, 1-2 times a month, and at least once a 
week). 

Table 4-11 shows that among fourth-graders who said they never or hardly ever wrote to friends or 
family using the Internet, 49 percent did not accept any automated spelling corrections, compared to 40 
percent among students who said they wrote to friends or family using the Internet at least once a 
week. Conversely, a higher percentage of students (37 percent) who said they wrote to friends or family 
using the Internet at least once a week accepted automated spelling corrections 4 times or more, 
compared to 30 percent among students who never or hardly ever did so. 

Table 4-11. Percentage distribution of fourth-grade students who wrote to friends or family using the 
Internet, by how often they wrote to friends or family and number of automated spelling 
corrections accepted: 2012 


Number of automated 
spelling corrections 
accepted 

How often students wrote to friends or family using the 
Internet 

All students 

Never or 
hardly ever 

1-2 times a month or a 
few times a year 

At least once a 
week 

All students 

44 

32 

24 

100 

0 

49 

45 

40 

46 

1 

9 

9 

10 

9 

2 

6 

7 

6 

6 

3 

6 

5 

6 

6 

>4 

30 

35 

37 

33 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


Students who said that they wrote to friends or family using the Internet more often appeared to be 
more likely to use the backspace key. For instance, 17 percent of students who said they wrote to 
friends or family using the Internet at least once a week used the backspace key more than 200 times, 
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compared to 13 percent of students who wrote to friends or family using the Internet 1-2 times a 
month or a few times a year and 8 percent of students who never or hardly ever wrote to friends or 
family using the Internet (table 4-12). In addition, a higher percentage of students (62 percent) who 
never or hardly ever wrote to friends or family using the Internet used the backspace key no more than 
100 times during the pilot assessment, compared to 52 percent of students who wrote to friends or 
family using the Internet 1-2 times a month or a few times a year and 44 percent of students who wrote 
to friends or family using the Internet at least once a week. The correlation between the frequency of 
use of the backspace key and the frequency of students' writing to friends or family using the Internet 
was substantive (r= .16). 

Table 4-12. Percentage distribution of fourth-grade students who wrote to friends or family using the 
Internet, by how often they wrote to friends or family and number of uses of backspace key: 
2012 



How often students wrote to friends or family using the Internet 

All students 

Number of uses 
of backspace key 

Never or 
hardly ever 

1-2 times a month or a few times a 

year 

At least once a 
week 

All students 

44 

32 

24 

100 

0 

# 

# 

# 

# 

1-100 

62 

52 

44 

54 

101-200 

29 

34 

39 

33 

>201 

8 

13 

17 

12 


# Rounds to zero. 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

Tables of the percentage of students who used each editing tool by how often they used the Internet to 
write to friends or family are available in appendix D as follows: 

• table D-10-al and table D-10-a2 for the language-related editing tools; 

• table D-10-b for the emphasis-related editing tools; and 

• table D-10-c for the revision-related editing tools. 

Writing on the computer for school assignments 

In the student questionnaire, fourth-grade students reported how much time they spent writing on a 
computer for school assignments in a day. About 34 percent of the students said they spent no time, 19 
percent said 10 minutes, 26 percent said half an hour, 13 percent said one hour, and 8 percent reported 
more than one hour per day (table 4-13). The amount of time spent writing on the computer for school 
assignments was not substantively correlated to any of the 12 editing tools, and the percentage of 
students who used each editing tool was very similar across all frequencies of use. 
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Tables of the percentage of students who used each editing tool by the amount of time they spent 
writing on the computer for school assignments are available in appendix D as follows: 

• table D-ll-al and table D-ll-a2 for the language-related editing tools; 

• table D-ll-b for the emphasis-related editing tools; and 

• table D-ll-c for the revision-related editing tools. 


Table 4-13. Percentage distribution of fourth-grade students who wrote on the computer for school 
assignments, by amount of time spent per day: 2012 




Amount of time spent per day 



None 

10 min 30 min 1 hour More than 1 hour 

All students 

All students 

34 

19 26 13 8 

100 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


3. The relationship between writing performance and prior exposure 
to writing on computer 

It was hypothesized that prior exposure to writing on the computer is positively related to writing 
performance. The relationship between the seven contextual variables related to prior exposure to 
writing on the computer and writing performance was examined in two ways: (1) the relationship 
between writing performance level and the degree of prior exposure, and (2) differences in raw writing 
scores by the degree of prior exposure. Only one of the seven prior-exposure variables was found to be 
substantively correlated to writing performance: access to the Internet at home. Four prior-exposure 
variables were observed to have some relationship with writing performance: students receiving 
computer-based writing assignments, looking for information on the Internet to include in their writing, 
writing to their friends or family using the Internet, and writing on the computer for school assignments. 
The remaining two prior-exposure variables— instruction in keyboarding and word processing and taking 
computer-based writing tests with an extended constructed-response component— did not have any 
relationship with writing performance. 

Instruction in keyboarding and word processing 

As shown in table 4-14, most fourth-grade students (88 percent) received some instruction in 
keyboarding and word processing. The score difference between those who received instruction in 
keyboarding and word processing and those who did not was 0.2, with an effect size of .17. This effect 
disappeared when the relationship was examined by performance level. 
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Table 4-14. Percentage and average raw scores of fourth-grade students whose teachers and staff did or 
did not spend time instructing students in keyboarding and word processing: 2012 



Percentage 


Average raw scores 


Score difference 


No 

Yes 

No 

Yes 

(Yes - No) 

All students 

12 

88 

2.76 

2.96 

0.20 


NOTE: Detail may not sum to totals because of rounding. Average raw score is reported on the scale of 1 to 6. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


The percentage and average raw scores of fourth-grade students by whether or not they received 
keyboarding and word processing instructions can be found in appendix D (table D-12). 

Receiving computer-based writing assignments 

Six percent of fourth-grade students reported having received any computer-based writing assignments 
once a week; 31 percent reported not having received any computer-based writing assignments; and 63 
percent reported having received such an assignment 1-2 times a month or a few times a year (table 4- 
15). In general, lower percentages of the higher performing students than the lower performing 
students had never or hardly ever received a computer-based writing assignment. For example, 26 
percent of high-performing students had never or hardly ever received a computer-based writing 
assignment, compared to 35 percent of low-performing students. However, the correlation between 
performance level and the frequency of having received any computer-based writing assignments was 
not substantive (r = . 07). 

The score difference between students who did not receive any computer-based writing assignment 
(average score = 1.80) and those who received such an assignment at least once a week (average score = 
2.12) was .32, with an effect size of .27. However, when the score difference was examined by 
performance level, the effects seemed to exist only for the high-performing student group. The average 
score of high-performing students who had never or hardly ever received a computer-based writing 
assignment was substantively lower than the scores of students who had received a computer-based 
writing assignment 1-2 times a month or less or once a week, with effect sizes of .20 and .22, 
respectively. 
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Table 4-15. Percentage and average raw scores of fourth-grade students, by how often they received 
computer-based writing assignments and performance level: 2012 




Percentage 


Average raw scores 


Performance 

level 

Never or 
hardly ever 

1-2 times a 
month or 
less 

Once a week 

Never or 
hardly ever 

1-2 times a 
month or less 

Once a 
week 

All students 

31 

63 

6 

2.82 

3.01 

3.14 

Low performing 

35 

59 

6 

1.40 

1.38 

1.44 

Middle 

performing 

31 

63 

6 

2.85 

2.90 

2.92 

High-performing 

26 

65 

9 

4.66 

4.77 

4.78 


NOTE: Detail may not sum to totals because of rounding. Average raw score is reported on the scale of 1 to 6. The 
low, middle, and high performance levels represent the bottom 20 percent, middle 60 percent, and top 20 percent 
of student performance, respectively. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

Computer-based writing tests with an extended constructed-response 
component 

Most fourth-grade students (84 percent) had not taken a computer-based writing test with an extended 
constructed-response component (table 4-16). This finding was similar across all three performance 
levels. The percentage of students by whether or not having taken computer-based writing tests with an 
extended construct-response component by performance level can be found in appendix D (table D-13). 

Table 4-16. Percentage and average raw scores of fourth-grade students, by whether or not they took 
computer-based writing tests with an extended constructed-response component: 2012 



Percentage 

Average raw scores 

Score difference 


No Yes 

No Yes 

(Yes - No) 

All students 

84 16 

2.96 2.92 

-0.04 


NOTE: Detail may not sum to totals because of rounding. Average raw score is reported on the scale of 1 to 6. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

Internet access at home 

Even though students can use the Internet at school or in a public library, access to the Internet at home 
might be more directly related to their use of the Internet for writing. Overall, about three-quarters of 
fourth-grade students (77 percent) responded that they had Internet access at home (table 4-17). There 
was a positive relationship between performance level and the percentage of students who had Internet 
access at home (r = .31). Specifically, 94 percent of students at the high performance level had access, 
compared to only about half of the students (52 percent) at the low performance level. The score 
difference of 0.98 between those with access to the Internet at home (3.19) and those without access 
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(2.21) was substantive, with an effect size of .87. The score difference by Internet access was 
substantive for each performance level as well, with effect sizes of .37 (for the low performance 
level), .49 (for the middle performance level), and .39 (for the high performance level). 

Table 4-17. Percentage and average raw scores of fourth-grade students, by Internet access at home and 
performance level: 2012 


Performance level - 

Percentage 

Average raw scores 

Score difference 
(Yes - No) 

No 

Yes 

No 

Yes 

All students 

23 

77 

2.21 

3.19 

0.98 

Low performing 

48 

52 

1.32 

1.45 

0.13 

Middle performing 

22 

78 

2.65 

2.95 

0.30 

High-performing 

6 

94 

4.53 

4.76 

0.23 


NOTE: Detail may not sum to totals because of rounding. Average raw score is reported on the scale of 1 to 6. 

The low, middle, and high performance levels represent the bottom 20 percent, middle 60 percent, and top 20 
percent of student performance, respectively. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment 

Using the Internet to get information for writing 

Overall, about one-third of fourth-grade students (32 percent) responded that they had never or hardly 
ever used the Internet to search for information to include in their writing (Table 4-18). About half of the 
students reported they used the Internet for this purpose 1-2 times a month or a few times a year and 
17 percent of students reported they used the Internet for this purpose once a week. 

A higher percentage of the low-performing than the high-performing students reported having never or 
hardly ever used the Internet to look for information to include in their writing (34 vs. 25 percent). 
Conversely, a lower percentage of the low-performing than the high-performing students reported 
having done so 1-2 times a month or a few times a year (46 vs. 58 percent). However, the correlation 
between how often students used the Internet to look for information to include in their writing and 
performance level was not substantive (r = .03). 

The average score of students who responded "1-2 times a month or a few times a year" was 3.10, 
which was 0.24 points higher than the average score of those who responded "never or hardly ever" and 
0.17 points higher than the average score of those who responded "once a week." The score difference 
of 0.24 has a small effect size of .20. When examined by performance level, there was no substantive 
score difference by the frequency with which students used the Internet to search for information to 
include in their writing. 
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Table 4-18. Percentage and average raw scores of fourth-grade students, by how often they looked for 
information on the Internet to include in their writing and performance level: 2012 


Performance 

level 


Percentage 


Average raw scores 


Never or 
hardly ever 

1-2 times a 
month or a 
few times a 
year 

Once a 
week 

Never or 
hardly ever 

1-2 times a 
month or a 
few times a 
year 

Once a 
week 

All students 

32 

51 

17 

2.86 

3.10 

2.93 

Low 

34 

46 

21 

1.39 

1.42 

1.39 

Middle 

31 

51 

18 

2.85 

2.92 

2.90 

High 

25 

58 

17 

4.70 

4.76 

4.76 


NOTE: Detail may not sum to totals because of rounding. Average raw score is reported on the scale of 1 to 6. The 
low, middle, and high performance levels represent the bottom 20 percent, middle 60 percent, and top 20 percent 
of student performance, respectively. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

Using the Internet to write to friends or family 

As shown in table 4-19, about 44 percent of fourth-grade students responded "never or hardly ever" 
when asked how often they wrote to their friends or family using the Internet. A lower percentage of 
the high-performing students (35 percent) than the middle- and low-performing students (46 and 44 
percent, respectively) reported never or hardly ever writing to their friends or family using the Internet. 
Conversely, a higher percentage of the high-performing students (30 percent) than the middle- and low- 
performing students (23 percent for each) reported writing to their friends or family using the Internet 
once a week. 

The average score of those who responded "never or hardly ever" was 2.89, which was lower than the 
average scores of those who responded "1-2 times a month or a few times a year" (3.07) and "once a 
week" (3.08). The score differences (0.18 and 0.18) have small effect sizes of .15 and .16, respectively. 
There was little difference in average scores between these two groups of students when examined by 
performance level. 
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Table 4-19. Percentage and average raw scores of fourth-grade students, by how often they wrote to 
their friends or family using the Internet and performance level: 2012 




Percentage 


Average raw scores 


Performance level 

Never or 
hardly ever 

1-2 times a 
month or a 
few times a 
year 

Once a week 

Never or 
hardly ever 

1-2 times a 
month or a 
few times a 

year Once a week 

All students 

44 

32 

24 

2.89 

3.07 

3.08 

Low 

44 

33 

23 

1.39 

1.40 

1.43 

Middle 

46 

31 

23 

2.86 

2.92 

2.91 

High 

35 

35 

30 

4.73 

4.78 

4.72 


NOTE: Detail may not sum to totals because of rounding. Average raw score is reported on the scale of 1 to 6. The 
low, middle, and high performance levels represent the bottom 20 percent, middle 60 percent, and top 20 percent 
of student performance, respectively. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

Writing on the computer for school assignments 

Overall, about one-third (34 percent) of fourth-grade students responded that they had not spent any 
time writing on the computer for school assignments (table 4-20). There appeared to be a negative 
association between performance level and the amount of time spent writing on the computer for 
school assignments beyond an hour. Whereas there was no substantive difference between low- 
performing students (27 percent) and middle- and high-performing students (33 percent) in the 
percentage of those who had not spent any time writing on the computer for school assignments, a 
higher percentage of low-performing students (12 percent) than high-performing students (5 percent) 
had done so for more than one hour per day. However, the seemingly negative association was not 
substantive (r = -.06). Students who had spent more than 1 hour writing on the computer for school 
assignments had an average writing score (2.57) that was substantively lower than the scores of 
students who had spent less time writing on the computer (2.90 to 3.08). However, the pattern of 
scores differed by performance level. For example, at the low performance level, those who responded 
"30 minutes" had a substantively higher average score (1.43) than those who responded "1 hour" (1.38), 
while at the middle performance level, those who responded "30 minutes" had an average score (2.89) 
that was not substantively different from the score for those who responded "1 hour" (2.86). 
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Table 4-20. Percentage and average raw scores of fourth-grade students, by how much time they spent 
writing on the computer for school assignments and performance level: 2012 


Percentage Average raw scores 


Performance 

level 

None 

10 

min 

30 

min 

1 

hour 

More than 
1 hour 

None 

10 

min 

30 

min 

1 

hour 

More than 
1 hour 

All students 

34 

19 

26 

13 

8 

3.05 

3.02 

3.08 

2.90 

2.57 

Low 

27 

19 

26 

16 

12 

1.44 

1.39 

1.43 

1.38 

1.33 

Middle 

33 

19 

26 

14 

8 

2.91 

2.92 

2.89 

2.86 

2.78 

High 

33 

19 

30 

13 

5 

4.74 

4.75 

4.81 

4.68 

4.61 


NOTE: Detail may not sum to totals because of rounding. Average raw score is reported on the scale of 1 to 6. The 
low, middle, and high performance levels represent the bottom 20 percent, middle 60 percent, and top 20 percent 
of student performance, respectively. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 
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This chapter examines students' preference for mode of writing (i.e., on computer, on paper, or does 
not matter) in relation to text length, the use of editing tools, and writing performance on the 2012 
NAEP computer-based pilot writing assessment. It was hypothesized that text length, the use of editing 
tools, and writing performance differ by preference for mode of writing. This chapter also examines 
students' preference for mode of writing in relation to prior exposure to writing on the computer and 
selected demographic variables. It was hypothesized that preference for mode of writing differs by prior 
exposure as well as by these demographic variables. These relationships are illustrated in figure 9 by the 
blue arrows and boxes. 

Figure 9. Conceptual model of the relationship between fourth-graders' writing performance and various 
associated factors 


Demographic 

Variables 



Prior Exposure to Writing on Computer 



Preference 
for Mode of 
Writing 



Writing Performance 
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1. The relationship between text length and preference for mode of 
writing 

As shown in table 5-1, the average length of responses by those who preferred writing on paper was 94 
words, compared to 116 words for those who preferred writing on the computer and 115 words for 
those who said the testing mode did not matter. As hypothesized, there was a difference in text length 
by preference for mode of writing. 

Table 5-1. Average, minimum, and maximum word counts of fourth-grade students, by mode preference 
on writing test: 2012 


Mode preference 


Word count 



Average 

Minimum 

Maximum 

Paper 

94 


0 

503 

Computer 

116 


0 

652 

Does not matter 

115 


0 

586 


NOTE: Boxplot for the distribution of word count by mode preference on writing test in Appendix F (figure F-7). 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


2. The relationship between editing tools and preference for mode of 
writing 

The relationship between the use of editing tools and preference for mode of writing was examined for 
the 12 editing tools. For six of these tools— spellcheck, accepting spelling corrections, thesaurus, bold, 
Italics, and backspace— differences were found in frequency of use across the three preferences for 
mode of writing. No differences were found for the remaining six tools. 

Language-related 

Spellcheck and accepting spelling corrections: As shown in table 5-2 , students who said they 
preferred writing on paper were more likely not to use spellcheck (34 percent) than were those who 
said they preferred writing on a computer (22 percent) or those who said the testing mode did not 
matter (23 percent). This pattern was also observed for accepting spelling corrections: a substantively 
higher percentage of students who preferred writing on paper (59 percent) did not accept any spelling 
corrections than did those who preferred writing on the computer (42 percent) or those who said the 
testing mode did not matter (44 percent). The opposite was true for the use of spellcheck 4 or more 
times: 43 percent of those who preferred writing on the computer and 41 percent of those who said the 
testing mode did not matter used spellcheck 4 or more times, compared to 31 percent of those who 
preferred writing on paper. 
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Table 5-2. Percentage distribution of fourth-grade students, by mode preference on writing test and 
number of uses of spellcheck tools: 2012 



Mode preference on writing test 


Number of uses of spellcheck tools 

Paper 

Computer 

Does not matter 

Spellcheck 

0 

34 

22 

23 

1 

15 

14 

15 

2 

11 

11 

12 

3 

9 

10 

11 

>4 

31 

43 

41 

Accepted automated spelling corrections 

0 

59 

42 

44 

1 

8 

9 

10 

2 

5 

6 

7 

3 

5 

6 

6 

>4 

23 

37 

33 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


Thesaurus and thesaurus replacement: As shown in table 5-3 , students who said they preferred 
writing on paper were more likely not to use the thesaurus tool (67 percent) than were those who said 
they preferred writing on a computer (55 percent) or those who said the testing mode did not matter 
(57 percent). There were no substantive differences in using the thesaurus replacement tool by 
preference for mode of writing. The percentage distribution of students by the frequency of using the 
thesaurus replacement tool can be found in appendix E (table E-l). 

Table 5-3. Percentage distribution of fourth-grade students, by mode preference on writing test and 
number of uses of thesaurus tool: 2012 


Number of uses of thesaurus tool 


Mode preference on writing test 


Paper 

Computer 

Does not matter 

Thesaurus 




0 

67 

55 

57 

1 

23 

31 

30 

>2 

9 

14 

13 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 
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Emphasis-related 

As shown in table 5-4, students who said they preferred writing on paper were more likely not to use 
the bold tool (69 percent) than were those who said they preferred writing on a computer (57 percent) 
or those who said the testing mode did not matter (60 percent). This pattern was also observed in the 
use of italics: a substantively higher percentage of students who preferred writing on paper (77 percent) 
did not use italics than did those who preferred writing on a computer (66 percent) or those who said 
the testing mode did not matter (69 percent). 

The percentage distribution for the frequency of use of the underline tool was similar across the three 
preferences for modes of writing and can be found in appendix E (table E-2). 

Table 5-4. Percentage distribution of fourth-grade students, by mode preference on writing test and 
number of uses of bold and italics: 2012 




Mode preference on writing test 


Number of uses of emphasis tools 

Paper 

Computer 

Does not matter 

Bold 

0 

69 

57 

60 

1 

14 

17 

18 

2 

7 

9 

8 

>3 

11 

17 

14 

Italics 

0 

77 

66 

69 

1 

11 

14 

14 

2 

5 

7 

7 

>3 

8 

13 

10 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


Revision-related 

As shown in table 5-5, differences were observed in the category of using the backspace key 1-100 
times: a higher percentage of students who preferred writing on paper were in this category (62 
percent) than were students who preferred writing on the computer (52 percent) or who did not have a 
preference (53 percent). Conversely, students who preferred writing on the computer were more like to 
use the backspace key 101-200 times (35 percent) than were those who preferred writing on paper (28 
percent). 

The percentage distribution for the frequency of use of the delete, cut, copy, and paste tools was similar 
across the three preferences for mode of writing and can be found in appendix E (table E-3). 
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Table 5-5. Percentage distribution of fourth-grade students, by mode preference on writing test and 
number of uses of the backspace key: 2012 


Number of uses of revision 

Mode preference on writing test 


tool 

Paper 

Computer 

Does not matter 

Backspace 




0 

1 

# 

# 

1-100 

62 

52 

53 

101-200 

28 

35 

34 

>201 

10 

13 

12 


# Rounds to zero. 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

3. Relationship between prior exposure to writing on the computer 
and preference for mode of writing 

The relationship between prior exposure to writing on the computer and preference for mode of writing 
was examined for seven contextual variables related to prior exposure. Three of the seven prior- 
exposure variables were found substantively related to the preference for mode of writing: Internet 
access at home, writing to family and friends using the Internet, and writing on the computer for school 
assignments. 

Internet access at home 

There was a substantive difference in fourth-grade students' preference for mode of writing by whether 
or not they had access to the Internet at home. For example, 25 percent of students who did not have 
access to the Internet preferred writing on paper (table 5-6), compared to 16 percent of students who 
had access. However, the percentage of students (47 percent) who had access to the Internet preferred 
writing on the computer was not substantively different from to the percentage of those who did not 
have access (41 percent). 

Table 5-6. Percentage distribution of fourth-grade students with different mode preferences on writing 
test, by access to the Internet at home: 2012 


Access to the Internet at home 


Mode preference on writing test 

No 

Yes 

All students 

Paper 

25 

16 

18 

Computer 

41 

47 

46 

Does not matter 

33 

37 

36 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 
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Using the Internet to write to friends or family 

In general, the more often students wrote to friends or family using the Internet, the more likely they 
were to prefer writing on the computer. For example, 55 percent of students who wrote to friends or 
family using the Internet at least once a week preferred writing on the computer, compared to 41 
percent of those who never or hardly ever did so (table 5-7). Conversely, a lower percentage of students 
who wrote to friends or family using the Internet at least once a week (13 percent) preferred writing on 
paper, compared to 21 percent of those who never or hardly ever did so. 

Table 5-7. Percentage distribution of fourth-grade students with different mode preferences on writing 
test, by how often they wrote to friends or family using the Internet: 2012 


How often students wrote to friends or family using the Internet 
Mode preference on 1-2 times a month All 


writing test 

Never or hardly ever 

or less often 

At least once a week 

students 

Paper 

21 

17 

13 

18 

Computer 

41 

46 

55 

46 

Does not matter 

38 

36 

33 

36 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


Writing on the computer for school assignments 

There seems to be some differences in preference for mode of writing by the amount of time that 
students spent writing on the computer for school assignments each day. For example, the percentage 
of students who preferred writing on the computer was 7 percentage points higher for those who spent 
more than 1 hour per day writing on the computer for school assignments (51 percent) than for those 
who spent no time (44 percent) (table 5-8). Flowever, the percentage of students who preferred writing 
on paper did not differ very much by the amount of time spent per day on the computer for school 
assignments, ranging from 16 percent for 1 hour to 19 percent for no time. 

Table 5-8. Percentage distribution of fourth-grade students with different mode preferences on writing 
test, by how much time per day they spent writing on the computer for school assignments: 
2012 



How much time spent writing on computer per day 

All students 

Mode preference on writing 
test 

None 

10 min 

30 min 

1 hour 

More than 
1 hour 

Paper 

19 

18 

17 

16 

17 

18 

Computer 

44 

44 

47 

49 

51 

46 

Does not matter 

37 

37 

37 

35 

32 

36 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 
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Prior-exposure variables with no relationship to preference for mode of writing 

Several prior-exposure variables did not seem to be related to students' preference for mode of writing: 
instruction in keyboarding and word processing, computer-based writing assignments, computer-based 
writing tests with an extended constructed-response component, and using the Internet to get 
information for writing. Tables on these variables are available in appendix E as follows: the percentage 
distribution of students' preferred mode of writing by whether or not their schools spent time 
instructing them in keyboarding and word processing (table E-4); the percentage distribution of 
students' preferred mode of writing by the frequency with which they received computer-based writing 
assignments (table E-5); the percentage distribution of students' preferred mode of writing by the 
frequency with which they took tests with an extended constructed- response component (table E-6); 
and the percentage distribution of students' preferred mode of writing by the frequency with which 
they used the Internet to look for information to include in their writing (table E-7). 

4. Relationship between writing performance and preference for 
mode of writing 

Most fourth-grade students (84 percent) had not had the experience of taking a NAEP-like writing 
assessment on a computer. Nevertheless, almost half of the students (46 percent) indicated that they 
would prefer taking a writing assessment on a computer, while less than one-fifth (18 percent) would 
prefer taking such an assessment on paper. The remaining students (36 percent) indicated that the 
testing mode did not matter. 

As hypothesized, there was an association between preference for mode of writing and performance 
level, with the percentage of students who preferred writing on paper decreasing as the performance 
level increased (table 5-9). Specifically, 11 percent of students at the high performance level preferred 
writing on paper, compared to 17 percent of students at the middle performance level and 27 percent 
of students at the low performance level. 

A score comparison across the three writing preference groups shows that students who indicated that 
the testing mode did not matter had the highest average score (3.15), followed by students who 
preferred writing on the computer (3.01) and students who preferred writing on paper (2.65). The score 
differences were substantive, with medium effect sizes of .30 and .41, respectively. At the high 
performance level, there was a substantive difference in scores between students who preferred writing 
on paper and those without a preference for testing mode, with an effect size of .2. 
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Table 5-9. Percentage and average raw writing scores of fourth-grade students, by mode preference on 
writing test and performance level: 2012 




Percentage 


Average raw score 


Performance level 

Paper 

Computer 

Does not 
matter 

Paper 

Computer 

Does not 
matter 

All students 

18 

46 

36 

2.65 

3.01 

3.15 

Low performing 

27 

44 

30 

1.38 

1.43 

1.40 

Middle performing 

17 

47 

36 

2.83 

2.90 

2.92 

High-performing 

11 

45 

44 

4.67 

4.73 

4.78 


NOTE: Detail may not sum to totals because of rounding. Average raw score is reported on the scale of 1 to 6. The 
low, middle, and high performance levels represent the bottom 20 percent, middle 60 percent, and top 20 percent 
of student performance, respectively. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

5. Relationship between demographic characteristics and preference 
for mode of writing 

The relationship between preference for mode of writing and demographic characteristics was 
examined for race/ethnicity (White, Black, Hispanic, and Asian), gender, National School Lunch Program 
(NSLP) eligibility, English language learners (ELL) status, and student disability (SD) status. Three of these 
characteristics— gender, ELL status, and SD status— were found related to a preference for mode of 
writing. In the subgroups defined by these characteristics, a higher percentage of the students who are 
historically lower performing preferred writing on paper. For example, a higher percentage of the SD 
students (28 percent) than the non-SD students (17 percent) indicated a preference for writing on 
paper. 

There was little variation in preference for writing on paper across the four racial/ethnic groups and 
little variation in preference for writing on the computer across Whites, Blacks, and Hispanics. Similarly, 
the variation in the percentage of the NSLP-eligible and non-eligible students who preferred writing on 
paper was not substantive. 
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Table 5-10. Percentage distribution of fourth-grade students, by mode preference on writing test and 
selected subgroups: 2012 





Percentage 


Subgroup 


Paper 

Computer 

Does not matter 


White 

17 

46 

37 


Black 

20 

48 

32 

Race/ethnicity 


Hispanic 

19 

46 

35 


Asian 

18 

39 

44 


Male 

21 

45 

34 

Gender 


Female 

15 

47 

38 


Not eligible 

16 

44 

39 

NSLP eligibility 


Eligible 

19 

48 

33 


Not ELL 

17 

46 

37 

ELL status 


ELL 

24 

45 

32 


Not SD 

17 

47 

37 

SD status 


SD 

28 

41 

31 


NOTE: Detail may not sum to totals because of rounding. For race/ethnicity, data are reported only for the four 


major groups. Black includes African American and Hispanic includes Latino. Race categories exclude Hispanic origin. 
For NSLP eligibility, the "information not available" category is not reported. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 
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This chapter examines the relationship between prior exposure to writing on the computer and 
demographic characteristics. The relationship was examined for seven variables related to prior 
exposure to writing on the computer and five demographic variables— race/ethnicity, gender, National 
School Lunch Program (NSLP) eligibility status, English language learners (ELL) status, and student 
disability (SD) status— and is illustrated in figure 10 by the blue arrow and boxes. For all seven prior- 
exposure variables, at least one demographic variable showed a different level of exposure to writing on 
the computer. 

Figure 10. Conceptual model of the relationship between fourth-graders' prior exposure to writing on the 
computer and demographic characteristics and various associated factors 


Demographic 

Variables 
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1. Instruction in keyboarding and word processing 

By race/ethnicity, a higher percentage of Asian students received instruction in keyboarding and word 
processing (94 percent) than did students from the other racial/ethnic groups, with percentages of 89 
percent for White students, 85 percent for Black students, and 86 percent for Hispanic students (table 6- 
1 ). 

Table 6-1. Percentage distribution of fourth-grade students, by whether or not teachers and staff spent 
time instructing them in keyboarding and word processing and selected subgroups: 2012 


Subgroup 


Percentage 

No 

Yes 


White 

11 

89 


Black 

15 

85 

Race/ethnicity 


Hispanic 

14 

86 


Asian 

6 

94 


Male 

12 

88 

Gender 


Female 

12 

88 


Not eligible 

10 

90 

NSLP eligibility 


Eligible 

13 

87 


Not ELL 

12 

88 

ELL status 


ELL 

11 

89 


Not SD 

12 

88 

SD status 


SD 

13 

87 


NOTE: Detail may not sum to totals because of rounding. For race/ethnicity, data are reported only for the four 
major groups. Black includes African American and Hispanic includes Latino. Race categories exclude Hispanic origin. 
For NSLP eligibility, the "information not available" category is not reported. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment 

2. Computer-based writing assignments 

The percentage of Asian students who had never or hardly ever received a computer-based writing 
assignment (24 percent) was lower than that of Hispanic students (39 percent) and Black students (31 
percent), and the percentage of White students (30 percent) was similar to that of Black students 
(table 6-2). There were no differences by gender or SD status, but there were differences by NSLP 
eligibility and ELL status. A higher percentage of NSLP-eligible and ELL students (35 percent and 39 
percent, respectively) than NSLP non-eligible and non-ELL students (28 percent and 31 percent, 
respectively) had never or hardly ever received a computer-based writing assignment. 
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Table 6-2. Percentage distribution of fourth-grade students, by how often they received computer-based 
writing assignments and selected subgroups: 2012 




Percentage 



Subgroup 


Never or hardly ever 

1-2 times a 
month or less 

Once a 
week 


White 

30 

66 

4 


Black 

31 

60 

9 

Race/ethnicity 


Hispanic 

39 

54 

7 


Asian 

24 

66 

10 


Male 

31 

63 

6 

Gender 


Female 

31 

62 

6 


Not eligible 

28 

66 

6 

NSLP eligibility 


Eligible 

35 

59 

6 


Not ELL 

31 

64 

6 

ELL status 


ELL 

39 

53 

8 


Not SD 

31 

63 

6 

SD status 


SD 

33 

60 

6 


NOTE: Detail may not sum to totals because of rounding. For race/ethnicity, data are reported only for the four 
major groups. Black includes African American and Hispanic includes Latino. Race categories exclude Hispanic origin. 
For NSLP eligibility, the "information not available" category is not reported. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

3. Computer-based writing tests with an extended constructed- 
response component 

As shown in table 6-3, Hispanic students (91 percent) were more likely not to have taken computer- 
based writing tests with an extended constructed-response component than were students in the three 
other racial/ethnic groups, all of which had similar percentages (83 or 84 percent). There were no 
substantive differences by gender, NSLP eligibility, or SD status. The percentage of ELL students who had 
taken computer-based writing tests with an extended constructed-response component (9 percent) was 
7 percentage points lower than that of corresponding non-ELL students (16 percent). 
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Table 6-3. Percentage distribution of fourth-grade students, by whether or not they took computer-based 
writing tests with an extended constructed-response component and selected subgroups: 
2012 


Subgroup 


Percentage 

No 

Yes 


White 

83 

17 


Black 

83 

17 

Race/ethnicity 


Hispanic 

91 

9 


Asian 

84 

16 


Male 

84 

16 

Gender 


Female 

84 

16 


Not eligible 

83 

17 

NSLP eligibility 


Eligible 

85 

15 


Not ELL 

84 

16 

ELL status 


ELL 

91 

9 


Not SD 

84 

16 

SD status 


SD 

84 

16 


NOTE: Detail may not sum to totals because of rounding. For race/ethnicity, data are reported only for the four 
major groups. Black includes African American and Hispanic includes Latino. Race categories exclude Hispanic origin. 
For NSLP eligibility, the "information not available" category is not reported. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

4. Internet access at home 

As shown in table 6-4, the percentage of Black and Hispanic students with Internet access at home (64 
percent) was about 20 percentage points lower than that of White and Asian students (85 and 87 
percent, respectively). Students in other low-performing groups also had lower percentages of Internet 
access at home than their respective counterparts: 65 percent for the NSLP eligible vs. 90 percent for 
the NSLP non-eligible students; 56 percent for the ELL vs. 79 percent for the non-ELL students; and 59 
percent for the SD vs. 80 percent for the non-SD students. 
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Table 6-4. Percentage distribution of fourth-grade students, by access to the Internet at home and 
selected subgroups: 2012 


Subgroup 


Percentage 

No 

Yes 


White 

15 

85 


Black 

36 

64 

Race/ethnicity 


Hispanic 

36 

64 


Asian 

13 

87 


Male 

23 

77 

Gender 


Female 

23 

77 


Not eligible 

10 

90 

NSLP eligibility 


Eligible 

35 

65 


Not ELL 

21 

79 

ELL status 


ELL 

44 

56 


Not SD 

20 

80 

SD status 


SD 

41 

59 


NOTE: Detail may not sum to totals because of rounding. For race/ethnicity, data are reported only for the four 
major groups. Black includes African American and Hispanic includes Latino. Race categories exclude Hispanic origin. 
For NSLP eligibility, the "information not available" category is not reported. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

5. Using the Internet to get information for writing 

An examination of how often students looked for information on the Internet to include in their writing 
by subgroup did not show a consistent pattern. There were no substantive differences by gender, NSLP 
eligibility, ELL status, or SD status. The percentage of students who looked for information on the 
Internet to include in their writing at least once a week was lower for White students (14 percent) than 
for Black (23 percent), Hispanic (21 percent), and Asian students (20 percent) (table 6-5). 
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Table 6-5. Percentage distribution of fourth-grade students, by how often they looked for information on 
the Internet to include in their writing and selected subgroups: 2012 





Percentage 


Subgroup 


Never or hardly ever 

1-2 times a month or 
less 

Once a week 


White 

35 

51 

14 


Black 

26 

50 

23 

Race/ethnicity 


Hispanic 

30 

49 

21 


Asian 

23 

57 

20 


Male 

34 

49 

18 

Gender 


Female 

30 

53 

17 


Not eligible 

31 

55 

14 

NSLP eligibility 


Eligible 

33 

47 

20 


Not ELL 

32 

51 

17 

ELL status 


ELL 

30 

47 

23 


Not SD 

31 

52 

17 

SD status 


SD 

33 

48 

19 


NOTE: Detail may not sum to totals because of rounding. For race/ethnicity, data are reported only for the four 
major groups. Black includes African American and Hispanic includes Latino. Race categories exclude Hispanic origin. 
For NSLP eligibility, the "information not available" category is not reported. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 

6. Using the Internet to write to friends or family 

An examination of how often students wrote to their friends or family using the Internet by subgroup 
did not reveal a consistent pattern (Table 6-6). 

While a higher percentage of male than female students (49 vs. 39 percent) and a lower percentage of 
Black than White students (38 vs. 46 percent) reported that they never or hardly ever wrote to friends 
or family using the Internet, no substantive differences were found by NSLP eligibility, ELL status, or SD 
status. 
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Table 6-6. Percentage distribution of fourth-grade students, by how often they wrote to their friends or 
family using the Internet and selected subgroups: 2012 





Percentage 


Subgroup 


Never or hardly ever 

1-2 times a month or 
less 

Once a week 


White 

46 

31 

23 


Black 

38 

32 

31 

Race/ethnicity 


Hispanic 

43 

34 

23 


Asian 

40 

36 

24 


Male 

49 

31 

20 

Gender 


Female 

39 

33 

29 


Not eligible 

45 

33 

23 

NSLP eligibility 


Eligible 

42 

32 

26 


Not ELL 

44 

32 

24 

ELL status 


ELL 

40 

35 

24 


Not SD 

44 

32 

24 

SD status 


SD 

44 

33 

23 


NOTE: Detail may not sum to totals because of rounding. For race/ethnicity, data are reported only for the four 


major groups. Black includes African American and Hispanic includes Latino. Race categories exclude Hispanic origin. 
For NSLP eligibility, the "information not available" category is not reported. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 


7. Writing on the computer for school assignments 

By race/ethnicity, the percentage of students who did not have prior experience writing on the 
computer for school assignments was higher for White students (40 percent) than for Black (26 
percent), Hispanic (24 percent), and Asian (23 percent) students; and higher for non-ELL students (35 
percent) than for ELL students (22 percent) (table 6-7). No substantive differences were found by 
gender, NSLP eligibility, or SD status. 
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Table 6-7. Percentage distribution of fourth-grade students, by amount of time per day spent writing on 
the computer for school assignments and selected subgroups: 2012 


Percentage 


Subgroup 


None 

10 min 

30 min 

1 hour 

More than 1 hour 


White 

40 

20 

24 

11 

6 


Black 

26 

17 

27 

17 

13 

Race/ethnicity 


Hispanic 

24 

17 

30 

18 

11 


Asian 

23 

17 

37 

17 

7 


Male 

35 

18 

25 

13 

8 

Gender 


Female 

32 

20 

27 

14 

8 


Not eligible 

37 

19 

27 

12 

5 

NSLP eligibility 


Eligible 

31 

18 

25 

15 

11 


Not ELL 

35 

19 

26 

13 

8 

ELL status 


ELL 

22 

18 

30 

19 

12 


Not SD 

34 

19 

26 

14 

7 

SD status 


SD 

30 

20 

24 

13 

12 


NOTE: Detail may not sum to totals because of rounding. For race/ethnicity, data are reported only for the four 
major groups. Black includes African American and Hispanic includes Latino. Race categories exclude Hispanic origin. 
For NSLP eligibility, the "information not available" category is not reported. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Pilot Writing Assessment. 
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Chapter 7. Summary, Discussion, and Potential 
Implications 

This chapter summarizes the conceptual framework that guided this study as well as the key findings. It 
also presents the conclusions that were made and the implications for policy, instruction, and further 
research that were drawn based on these findings. 

1. Summary and discussion 

Prior exposure to writing on the computer provides insight into fourth-grade students' acquisition of 
"facilitative skills" (i.e., keyboarding and word processing), which can impact writing performance. The 
data from this study indicated that some prior exposure related to writing on the computer is associated 
with text length, the use of the spellcheck and backspace key, and preference for mode of writing— all of 
which are associated with writing performance. 

Middle- and high-performing students, who are more likely to have prior exposure to writing on the 
computer, produced longer texts and had higher writing scores than low-performing students. Because 
their cognitive resources were presumably not directed at keyboarding, they could have been freed 
for deeper thinking, which often requires, for example, changing words via spellcheck and changing 
sentences via the backspace key when words and sentences do not work in light of the given audience, 
genre, or purpose of writing. 

When students produce shorter texts, they might not have a compelling reason to engage in an iterative 
writing process— that is, to revisit previously written text to correct mistakes using spellcheck and the 
backspace key or to adjust the order in which information is presented. Finally, because keyboarding 
likely consumes most of their attention, few cognitive resources are left for selection of more complex 
words and development of ideas in a manner that fulfills their audience's needs, the requirements of 
the genre, and the purpose of writing— all of which are essential to the quality of writing in NAEP. 

In support of the perceived challenges that these students encounter when keyboarding, an observer of 
the cognitive labs conducted for the 2012 NAEP fourth-grade writing assessment had this to say: 

"Fourth-graders' preferred typing style is a 'vulture stance' where they hover, hunched over 
the keyboard, and let one or two fingers descend from a height of 8 or 10 inches directly 
above the keyboard, straight down to poke the desired key or keys. A second, more relaxed 
mode involves only one hand. The other hand is propped up, next to the keyboard, by the 
elbow. The head is tipped over to the side of the keyboard and is supported by the non- 
typing hand. From this angle, the student can locate the desired keys, and let the other hand 
do the typing. About half of the students observed said that they would probably have 
written more if they had been writing using paper and pencil" (E. Greer, personal e-mail 
communication, May 31, 2013). 
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When asked about their preferred mode of writing in the 2012 NAEP Student Questionnaire, fourth- 
graders with the following characteristics were more likely to say that they would prefer to take a 
writing test on paper: students with no access to the Internet at home, students with less experience 
looking for information on the Internet to include in their writing (thus leading to less content 
knowledge for development of ideas), and students with less experience writing to friends and family 
using the Internet. A higher percentage of fourth-graders who were male, English language learners, 
and/or who had a disability also preferred to write on paper. 

The lack of a substantive relationship between writing scores and the use of the delete key, the revision 
tools, (cut, copy, paste) and the emphasis-related tools (bold, italic, underline), might seem puzzling. It 
is, however, probable that this is related to a very small percentage of fourth-graders (3 percent) having 
used the delete key (which, unlike the backspace key, deletes text to the right of the cursor). As for the 
other revision keys, it is likely that low- performing students did not have much text generated to the 
left of their cursor to revise. The percentage of fourth-graders using the other revision tools and the 
emphasis tools was also relatively small, suggesting that most fourth-graders might not have been 
familiar or comfortable with these editing functions. Low-performing students might have simply been 
playing with these features— e.g., making words bold— hence their seemingly negative relationship with 
writing scores. In support of this interpretation, the independent report of the NAEP computer-based 
writing usability study (Fulcrum IT 2011) said this: "Students were unaware of standard symbols for 
editing and formatting, such as icons for copy and paste, indent, bold, italics, and underline." 

Although the availability and use of computers has increased dramatically recently— both at home and 
at school— opportunities to benefit from such technology in composing text on the computer are not 
necessarily distributed equally across all students. The NAEP data indicate that low-performing students 
have less exposure to writing on the computer, both inside and outside of school, and that they produce 
shorter texts. When a student is only capable of producing a few sentences per hour on the computer in 
a NAEP-like assessment, he or she is likely to have difficulties composing a whole essay; part of that 
difficulty may have to do with simply finding the letters on the keyboard and the editing tools on the 
menu— taking time and cognitive resources away from critical thinking processes, such as finding the 
right word. 

2. Potential implications 

The NAEP item developers are charged with producing assessment tasks that match the NAEP target 
domain specifications. These measures do not distinguish whether poorer writing performance is due to 
deficiencies in facilitative skills (i.e., keyboarding and word processing in this study), target skills (i.e., 
language facility, development of ideas, and organization of ideas), or both. 

Policymakers and teachers are left with the task of equipping students with the resources and 
facilitative skills required for successful performance on assessments with a computer component. 
Strategies for maximizing the computer-based facilitative skills of fourth-grade students, especially low- 
performing students, may include classroom instruction on how to compose at the keyboard and take 
advantage of common computer editing functions, such as spellcheck. In this way, they can focus their 
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attention on cognitive processes, such as attention to precise word choices, varied sentences, and 
strong details and supporting evidence, rather than on typing processes and translating writing thoughts 
into keyboard presses. Moreover, this will help place them on an even playing field with high- 
performing fourth-grade students on the NAEP computer-based writing assessment, which reflects how 
today's students are expected to compose text on the computer at school and in the workplace. 

Secondary researchers may attempt to replicate the specific findings in this report in a greater range of 
situations and grade levels, formulate new hypotheses based on the patterns suggested by the 
nonsubstantive differences, and evaluate their relationships with the target measures in the 
assessment. For example, what is the relationship among different types of prior exposure to writing on 
the computer, such as the relationship between Internet access at home and using the Internet to get 
the information for writing? To what extent are facilitative skills different at grades 4, 8, and 12? To 
what extent are they different across other computer-based subject areas that NAEP assesses and 
across other computer devices, such as tablets? Is there a core set of school-, student-, and teacher- 
related survey questions or observed practices that best measure students' experience with 
keyboarding and word processing in and out of school? Is fluency with revision tools (backspace, delete, 
cut, copy, and paste) more relevant to the organization and development of ideas than to language 
skills? These research efforts are appropriate and interesting in themselves; however, they are also 
important in further estimating the potential impact of facilitative skill deficiencies on writing 
performance and in explaining why this is or is not the case. 

As for large-scale assessments such as NAEP, when students bring to the assessment the required 
facilitative skills described in this report, it fosters more valid information and interpretation about the 
level of students' target skills. This, in turn, may lead to more valid new baselines for showing how the 
writing skills of fourth-grade students, especially low-performing students, has changed over time. 
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Exact text of questions in the 2012 NAEP Writing Contextual Questionnaires 


Contextual questions related to writing on the computer 

Source 

Instruction on keyboarding and word processing 


By the end of this school year, approximately what percentage of teaching 
time will your teachers and staff have spent instructing fourth-grade 
students on keyboarding and word processing? 0 percent, 1-5 percent, 6-10 
percent, 11-25 percent, 26-50 percent, and over 50 percent 

School questionnaire 

Based on keyboarding instruction atyour school, which of the following best 
describes the keyboarding skills expected of your fourth-grade students: 

• No typing skill is expected because we do not give formal 
keyboarding instruction 

• Two-finger typing 

• Basic touch-typing 

• Rapid and accurate touch-typing 

Teacher questionnaire 

Instruction on word processing 


How often do you ask your students to do the following when you ask them 
to write about something : 

• Use the computer for drafting and revising their writing 

• Use the computer to complete writing that is started by hand 

• Use the Internet to get pictures, sound, or video files for their 
writing 

Teacher questionnaire 

Which of the following word processing tools are your fourth-grade students 
expected to be able to use for writing: copy/cut/paste, spell-check, 
thesaurus, formatting tools (e.g., bold, underline, italics), my fourth-grade 
students are not expected to use any of the word processing tools listed 
above 

Teacher questionnaire 
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Computer-based writing assignments 


How often do fourth-grade students in your school receive computer-based 
writing assignments? Never or hardly ever, a few times a year, once or twice 
a month, at least once a week 

School questionnaire 

In a day, about how much time do you spend writing on a computer for 
school assignments: None, about 10 minutes, about 30 minutes, about 1 
hour, more than 1 hour 

Student questionnaire 

Computer-based writing tests with an extended constructed- 
response component 

How often do fourth-grade students in your school take computer-based 
writing tests with an extended constructed-response component? Never or 
hardly ever, a few times a year, once or twice a month, at least once a week 

School questionnaire 

Internet access at home 

Do you have the following items at your home? Access to the Internet, clothes 
dryer just for your family, dishwasher, more than one bathroom, your own 
bedroom 

Student questionnaire 

Internet access for instruction 

Which statement best describes computer availability for your writing 
instruction? There is no computer for student use, there is one computer 
shared by all students, more than three students share one computer, two or 
three students share one computer, each student has a computer 

Teacher questionnaire 

Do you have access to the Internet when you teach writing to your students? 
Yes, always; yes, sometimes ; no 

Teacher questionnaire 

Using the Internet to get information for writing 

How often do you ask your students to do the following when you ask them 
to write about something: use the Internet to get information for their 
writing 

Teacher questionnaire 

For school this year, how often do you look for information on the Internet to 
include in your writing? Never or hardly ever, a few times a year, once or 
twice a month, at least once a week 

Student questionnaire 
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Using the Internet to write to friends or family 


How often do you write to your friends or family using the Internet— for 
example, e-mails, blogs, text messages, instant messages, or personal web 
pages: Never or hardly ever, once or twice a month, once or twice a week, 
every day or almost every day 

Student questionnaire 

Preference for taking a writing test 


How do you prefer to take a writing test? On computer, on paper, it does not 
matter 

Student questionnaire 


NOTE: It was not possible to analyze the teacher questions highlighted in blue because they were not linked to 
student performance. 
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Table B-l. Weighted school response rates for the 2012 NAEP grade 4 writing pilot assessment, by 
census region: 2012 


Census region 

Weighted school response rate 

National 

87 

Midwest 

98 

Northeast 

95 

South 

73 

West 

94 


NOTE: Preliminary weights which adjust for unequal probability of selection were used to calculate the 
weighted response rate. Public and private schools were combined in the calculation. Texas's refusal is 
the main source of school nonresponse. 


Table B-2. Unweighted student response rates for the 2012 NAEP grade 4 writing pilot assessment, by 
census region: 2012 


Census region 

Unweighted student response rate 

National 

95 

Midwest 

96 

Northeast 

94 

South 

95 

West 

96 


NOTE: Preliminary weights were not available for individual students. Therefore only unweighted 
student response rates can be provided. 
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Table C-l. Percentage distribution of fourth-grade students who used thesaurus editing tools with various 
frequencies, by performance level: 2012 


Number of uses 


Performance level 



of thesaurus 
editing tools 

Low 

Middle 

High 

All students 

Thesaurus 

0 

59 

58 

59 

59 

1 

28 

30 

28 

29 

>2 

13 

12 

13 

13 

Thesaurus 

replacement 

0 

94 

93 

93 

93 

1 

5 

6 

6 

6 

>2 

1 

1 

1 

1 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


• Nearly 60 percent of the fourth-grade students did not activate the thesaurus tool at all during 
the assessment. 

• Ninety three percent of the students did not use the thesaurus replacement. 

• There was little difference in using thesaurus-editing tools by performance level. For example, 
the percentage of the students at the low performing level who invoked the thesaurus two 
times or more (13 percent) was similar to those at the middle and high-performing levels (12 
percent and 13 percent, respectively). 
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Table C-2. Percentage distribution of fourth-grade students who used emphasis-related word processing 
tools with various frequencies, by performance level: 2012 




Performance level 


Number of uses of emphasis editing tools 

Low 

Middle 

High 

All students 

Bold 

0 

60 

61 

60 

60 

1 

17 

17 

16 

17 

2 

8 

8 

9 

8 

>3 

14 

15 

15 

15 

Italic 

0 

71 

70 

67 

69 

1 

14 

13 

13 

13 

2 

6 

6 

8 

6 

>3 

9 

11 

13 

11 

Underline 

0 

69 

71 

70 

70 

1 

17 

15 

15 

15 

2 

6 

7 

6 

6 

>3 

9 

8 

10 

8 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


• Sixty percent of fourth-graders did not use the bold feature. Percentage of fourth-graders who 
did not use the italics and of those who did not use underline feature was 69 percent and 70 
percent, respectively. 

• The percentage of fourth-grade students who used the bold feature 3 times or more was 15, 
followed by italics (11 percent) and underline (8 percent). 

• The frequency pattern of using the emphasis related editing tools for those at the low 
performing level was similar to those at the middle and high-performing levels. 
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Table C-3. Percentage distribution of fourth-grade students who used revision-related word processing 
tools with various frequencies, by performance level: 2012 


Number of 


Performance level 



uses of revision 
editing tools 

Low 

Middle 

High 

All students 

Delete 

0 

96 

97 

96 

97 

> 1 

4 

3 

4 

3 

Paste 

0 

83 

86 

89 

86 

> 1 

17 

14 

11 

14 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


• Among the fourth-graders, 97 percent did not use the "delete" at all. Over 85 percent of fourth- 
graders did not "paste". When examining those who did use delete or paste features by 
performance level, there was little difference in using the two revision-related editing tools by 
performance level. 

Table C-4. Average, minimum, and maximum word count of responses of fourth-grade students, by 
number of times of using spellcheck editing tool: 2012 


Number of uses of spellcheck editing 


Word count 



tool 

Average 

Minimum 

Maximum 

Spellcheck 





0 

105 


0 

586 

1 

107 


0 

430 

2 

110 


0 

652 

3 

115 


1 

594 

>4 

114 


0 

500 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


• Students who did not invoke the spellcheck at all wrote about 105 words on average, and 
students who invoked this editing tool 3 times or 4 or more wrote on average 115 and 114 
words, respectively. These differences of about 10 more words were not substantive. 
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Table C-5. Average, minimum, and maximum word count of responses of fourth-grade students, by 
number of times of using thesaurus editing tools: 2012 


Number of uses of thesaurus editing 


Word count 


tools 

Average 

Minimum 

Maximum 

Thesaurus 

0 

114 

0 

652 

1 

107 

0 

460 

>2 

102 

0 

410 

Thesaurus replacement 

0 

111 

0 

652 

1 

107 

3 

432 

>2 

107 

6 

340 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


• Students who never activated the thesaurus tool at all during the pilot assessment wrote about 
114 words on average and students who activated the tool once wrote about 107 words on 
average. Students who activated the tool 2 or more times wrote 102 words on average. 

• Students who never used the thesaurus replacement during the assessment also produced 
about 111 words and those who used once or those who used at least twice wrote about 107 
words. 
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Table C-6. Average, minimum, and maximum word count of responses of fourth-grade students, by 
number of times of using emphasis-related editing tools: 2012 


Number of uses of emphasis 


Word count 



editing tools 

Average 

Minimum 

Maximum 

Bold 

0 

112 


0 

518 

1 

109 


0 

652 

2 

106 


2 

586 

>3 

106 


0 

475 

Italic 

0 

111 


0 

594 

1 

109 


1 

572 

2 

109 


0 

573 

>3 

106 


0 

652 

Underline 

0 

112 


0 

594 

1 

106 


0 

481 

2 

104 


1 

573 

>3 

107 


0 

652 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment 


• For each of the emphasis-related editing tool, students who did not use it during the assessment 
wrote about 112 words, while students who used the tools 3 or more times wrote about 106 
words. However, the average word count of responses generally did not differ much by the 
number of times of use for each emphasis-related editing tool. 
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Table C-7. Average, minimum, and maximum word count of responses of fourth-grade students, by 
number of times of using revision-related editing tools: 2012 




Word count 


Number of uses of revision editing tools 

Average 

Minimum 

Maximum 

Delete 

0 

110 

0 

652 

> 1 

111 

0 

573 

Copy 

0 

112 

0 

652 

> 1 

99 

0 

500 

Paste 

0 

112 

0 

652 

> 1 

100 

0 

500 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


• There were no substantive differences in the length of responses between students who used 
and those who did not use each of revision-related editing tools, delete, copy, and paste. 
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1. The relationship between text length and prior exposure to writing 
on the computer 

Instruction on keyboarding and word processing 

Table D-l. Average, minimum, and maximum word count of responses of fourth-grade students, by 

whether or not teachers and staff have spent time instructing students on keyboarding and 
word processing: 2012 


Time spent on keyboarding and word processing? 

Average 

Word count 
Minimum 

Maximum 

No 

106 

0 

460 

Yes 

110 

0 

652 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


Translating writing thoughts into keyboard strokes would require additional cognitive demands for 
students with no instruction on keyboarding or word processing and would limit the length of text 
compared to students who had received such instruction. On average, students who attended schools 
where teachers and staff (as reported by school administrators) provided keyboarding and word 
processing instruction wrote responses of 110 words, while students whose teachers did not provide the 
instruction wrote responses of 106 words. Prior instruction on keyboarding and word processing does 
not make any substantive difference in the length of text. 

Computer-based writing assignments 

Table D-2. Average, minimum, and maximum word count of responses of fourth-grade students, by how 
often students received computer-based writing assignments: 2012 


How often students received computer-based writing 
assignments? 

Average 

Word count 
Minimum 

Maximum 

Never or hardly ever 

105 

0 

594 

1-2 times a month or a few times a year 

111 

0 

652 

Once a week 

124 

0 

493 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The average text length of responses was the longest for the students whose school reported their 
students having received computer-based assignments once a week (average word count = 124), 
followed by responses of students who were reported having had such assignments 1 to 2 times a month 
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or a few times a year (average word count = 111), and the shortest for the responses of students who 
reported never or hardly ever receiving any computer-based writing assignments (average word count = 
105). The correlation between the frequency of receiving computer-based writing assignments and text 
length was non-substantive (r= .06). The non-substantive relationship is most likely due to the large 
variation of word count within each frequency level of receiving computer-based writing assignments. 

Computer-based writing tests with an extended constructed-response 
component 

Table D-3. Average, minimum, and maximum word count of responses of fourth-grade students, by how 
often students take computer-based writing tests with an extended constructed-response 
component: 2012 


How often students received computer-based writing 
tests with an extended CR component? 

Average 

Word count 
Minimum 

Maximum 

Never or hardly ever 

109 

0 

652 

1-2 times a month or a few times a year 

110 

0 

573 

Once a week 

X 

t 

t 


t Not applicable, 
t Reporting standards not met. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 

In addition to computer-based writing assignments, the staff of participating schools also provided 
information about how often their fourth-grade students took computer-based writing tests with an 
extended constructed-response component. Students who had never received such test and those who 
had taken such test 1 to 2 times a month or a few times a year wrote approximately 110 words on 
average during the fourth-grade writing pilot assessment. The correlation between the frequency of 
receiving computer-based writing tests with an extended CR (constructed response) component and 
word counts is close to zero (r = -.006). 

Writing on the computer for school assignments 

Table D-4. Average, minimum, and maximum word count of responses of fourth-grade students, by how 
much time spent writing on the computer for school assignments: 2012 


How much time spent writing on the computer for 


Word count 


school assignments? 

Average 

Minimum 

Maximum 

None 

107 

0 

594 

10 min 

113 

0 

652 

30 min 

117 

0 

572 

1 hour 

116 

1 

586 

More than 1 hour 

103 

0 

573 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 
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Students who spent 10, 30 or 60 minutes writing on a computer for school assignment, on average, 
seem to have written longer texts (between 113 and 117 words on average) than students who did not 
spend any time writing assignments on computer (107 words). The difference of about 10 words was 
not substantive, however. The correlation between the amount of time spent writing on the computer 
for school assignments and word counts is close to zero (r = .03). 

2. The relationship between editing tools and prior exposure to 
writing on computer 

Instruction on keyboarding and word processing 

Language-related editing tools 

Table D-5-a. Percentage distribution of fourth-grade students who used the spellcheck with various 
frequencies, by whether or not they received instructions on keyboarding and word 
processing: 2012 


Number of uses of 

Instructions on keyboarding and word processing? 



spellcheck editing tools 

No 

Yes 

All students 

All students 

12 

88 

100 

Spellcheck 

0 

25 

25 

25 

1 

16 

14 

14 

2 

12 

11 

11 

3 

11 

10 

10 

>4 

37 

40 

40 

Accepted automated 
spelling corrections 

0 

49 

46 

46 

1 

10 

9 

9 

2 

6 

6 

6 

3 

5 

6 

6 

>4 

30 

33 

33 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The spellcheck: Prior exposure to instructions on keyboarding and word processing is not related to the 
frequency of fourth-graders using spellcheck or accepting automated spelling corrections during the 
fourth-grade writing pilot assessment. Table D-5-a shows that the percentage of students using 
spellcheck or accepting automated spelling corrections was similar with various frequencies between 
students who received instructions on keyboarding and word processing and those who did not. In other 
words, there was no substantive correlation between using the spellcheck or accepting automated 
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spelling corrections and prior exposure to instructions on keyboarding and word processing (r = .02 
and .03, respectively). 

Table D-5-b. Percentage distribution of fourth-grade students who used the thesaurus with various 
frequencies, by whether or not they received instructions on keyboarding and word 
processing: 2012 


Number of uses of thesaurus 

Instructions on keyboarding and word processing? 



editing tools 

No 

Yes 

All students 

All students 

12 

88 

100 

Thesaurus 

0 

57 

58 

58 

1 

30 

29 

29 

>2 

12 

13 

13 

Thesaurus replacement 

0 

93 

93 

93 

1 

6 

6 

6 

>2 

1 

1 

1 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The thesaurus: Similarly, the prior exposure to instructions on keyboarding and word processing is not 
related to the frequency of fourth-graders' use of thesaurus or thesaurus replacement during the 
fourth-grade writing pilot assessment (r = -.004 and -.006, respectively). Table D-5-b shows that a 
similar percentage of students invoked the thesaurus or the thesaurus replacement with various 
frequencies whether or not they received instructions on keyboarding and word processing. 
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Emphasis-related editing tools 

Table D-5-c. Percentage distribution of fourth-grade students who used emphasis-related editing tools 
with various frequencies, by whether or not they received instructions on keyboarding and 
word processing: 2012 


Number of uses of emphasis 

Instructions on keyboarding and word processing? 



editing tools 

No 

Yes 

All students 

All students 

12 

88 

100 

Bold 

0 

60 

60 

60 

1 

18 

17 

17 

2 

8 

8 

8 

>3 

14 

15 

15 

Italics 

0 

69 

69 

69 

1 

14 

13 

13 

2 

6 

7 

7 

>3 

10 

11 

11 

Underline 

0 

69 

70 

70 

1 

17 

15 

15 

2 

7 

6 

6 

>3 

7 

9 

8 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The same relationship is observed between prior exposure to instructions on keyboarding and word 
processing and the frequency of fourth-graders' use of emphasis-related editing tools during the fourth- 
grade writing pilot assessment (Table D-5-c). The pattern of using bold, italics or underline with various 
frequencies is similar between students who received instructions on keyboarding and word processing 
(based on the school administrators' report) and students who did not. (The absolute value of 
correlation (r) was smaller than 0.01 for all three associations.) 
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Revision-related editing tools 

Table D-5-d. Percentage distribution of fourth-grade students who used revision-related editing tools 
with various frequencies, by whether or not they received instructions on keyboarding and 
word processing: 2012 


Number of uses of revision 

Instructions on keyboarding and word processing? 



editing tools 

No 

Yes 

All students 

All students 

12 

88 

100 

Backspace 

0 

# 

# 

# 

1-100 

52 

55 

54 

101 - 200 

36 

33 

33 

>201 

12 

12 

12 

Delete 

0 

97 

96 

97 

> 1 

3 

4 

3 

Cut 

0 

78 

80 

80 

> 1 

22 

20 

20 

Copy 

0 

86 

86 

86 

> 1 

14 

14 

14 

Paste 

0 

86 

86 

86 

> 1 

14 

14 

14 


# Rounds to zero. 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 

As shown in table D-5-d, the pattern of using revision-related editing tools (i.e., backspace, delete, cut, 
copy, and paste) with various frequencies is also similar between students who received instructions on 
keyboarding and word processing and students who did not. (The absolute value of correlation (r) was 
smaller than .02 for all five associations.) 
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Receiving computer-based writing assignments 


Language-related editing tools 

Table D-6-a. Percentage distribution of fourth-grade students using spellcheck editing tool with various 
frequencies, by how often they received computer-based writing assignments: 2012 


Number of 

How often received computer-based writing assignments? 



uses of 
spellcheck 

Never or hardly 

1-2 times a month or a few times a 



editing tool 

ever 

year 

At least once a week 

All students 

All 

students 

31 

63 

6 

100 

Spellcheck 





0 

27 

24 

22 

25 

1 

15 

14 

15 

14 

2 

12 

11 

12 

11 

3 

9 

10 

11 

10 

>4 

37 

41 

39 

40 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The spellcheck: The pattern of using spellcheck with various frequencies is similar no matter how often 
students received computer-based writing assignments (table D-6-a). 

Table D-6-b. Percentage distribution of fourth-grade students using thesaurus editing tools with various 
frequencies, by how often they received computer-based writing assignments: 2012 


Number of uses of How often received computer-based writing assignments? 


thesaurus editing 
tools 

Never or 
hardly ever 

1-2 times a month or 
a few times a year 

At least once a week 

All students 

All students 

31 

63 

6 

100 

Thesaurus 

0 

58 

58 

62 

58 

1 

29 

29 

28 

29 

>2 

13 

13 

10 

13 

Thesaurus replacement 

0 

94 

93 

94 

93 

1 

5 

6 

6 

6 

>2 

1 

1 

# 

1 


# Rounds to zero 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 
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The thesaurus: The frequency with which students received computer-based writing assignments 
based on the school report was not associated with students' use of thesaurus and thesaurus 
replacement during the writing pilot assessment. Absolute value of correlation (r) was smaller than .015 
for both editing tools). 

Emphasis-related editing tools 

Table D-6-c. Percentage distribution of fourth-grade students using emphasis-related editing tools with 
various frequencies, by how often they received computer-based writing assignments: 2012 


Number of uses of 

How often received computer-based writing assignments? 



emphasis editing 
tools 

Never or 
hardly ever 

1-2 times a month or a 

few times a year At least once a week 

All students 

All students 

31 

63 

6 

100 

Bold 

0 

63 

59 

61 

60 

1 

18 

16 

17 

17 

2 

7 

9 

10 

8 

>3 

12 

16 

12 

15 

Italics 

0 

71 

68 

69 

69 

1 

14 

13 

12 

13 

2 

5 

7 

8 

7 

>3 

10 

12 

11 

11 

Underline 

0 

72 

69 

69 

70 

1 

14 

16 

18 

15 

2 

6 

6 

7 

6 

>3 

8 

9 

6 

8 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


Similarly, how often students received computer-based writing assignments based on the school report 
was not associated with students' use of emphasis-related editing tools during the writing pilot 
assessment (table D-6-c). (Absolute value of correlation (r) was lower than .05 for three associations.) 
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Revision-related editing tools 

Table D-6-d. Percentage distribution of fourth-grade students using revision-related editing tools with 
various frequencies, by how often they received computer-based writing assignments: 2012 


Number of uses of 

How often received computer-based writing assignments? 



revision editing 
tools 

Never or 
hardly ever 

1-2 times a month or 

a few times a year At least once a week 

All students 

All students 

31 

63 

6 

100 

Delete 

0 

97 

97 

96 

97 

> 1 

3 

3 

4 

3 

Cut 

0 

78 

80 

84 

80 

> 1 

22 

20 

16 

20 

Copy 

0 

87 

86 

87 

86 

> 1 

13 

14 

13 

14 

Paste 

0 

86 

86 

89 

86 

> 1 

14 

14 

11 

14 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The frequency of students' use of delete, cut, copy, and paste did not differ by frequency of receiving 
computer-based writing assignments based on the school report (table D-6-d). 
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Computer-based writing tests with an extended constructed-response 
component 

Language-related editing tools 

Table D-7-a. Percentage distribution of fourth-grade students using spellcheck editing tools with various 
frequencies, by how often they took computer-based writing tests with an extended 
constructed-response component: 2012 


Number of uses 

How often received computer-based writing tests with an extended 
constructed-response component? 


of spellcheck 
editing tools 

Never or hardly ever 

1-2 times a month or 
a few times a year 

All students 

All students 

84 

16 

100 

Spellcheck 

0 

25 

24 

25 

1 

14 

16 

14 

2 

11 

12 

11 

3 

10 

9 

10 

>4 

40 

41 

40 

Accepted 

automated 

spelling 

corrections 

0 

46 

46 

46 

1 

9 

11 

9 

2 

6 

5 

6 

3 

6 

5 

6 

>4 

32 

33 

32 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The spellcheck: Whether or not students had the experience of taking computer-based writing tests 
with an extended constructed-response component is not related to the frequency pattern of using the 
spellcheck tools, as shown in table D-7-a. A similar percentage of students who never or hardly ever 
took computer-based tests used the spellcheck editing tools with various frequencies during the writing 
pilot assessment, compared to students who received computer-based writing tests more frequently (r 
< .01 for both associations). 
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Table D-7-b. Percentage distribution of fourth-grade students using thesaurus editing tools with various 
frequencies, by how often they took computer-based writing tests with an extended 
constructed-response component: 2012 



How often received computer-based writing tests with an 
extended constructed-response component? 


Number of uses of 
thesaurus editing tools 

Never or hardly ever 

1-2 times a month or 
a few times a year 

All students 

All students 

84 

16 

100 

Thesaurus 

0 

58 

59 

58 

1 

29 

29 

29 

>2 

13 

12 

13 

Thesaurus 

replacement 

0 

93 

93 

93 

1 

6 

6 

6 

>2 

1 

1 

1 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The thesaurus: The pattern of using thesaurus editing tools does not differ between students who had 
the experience of taking computer-based writing tests with an extended component and those who did 
not (table D-7-b), which is similar to spellcheck editing tools. (Absolute value of correlation (r) was larger 
than .01 for both associations.) 
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Emphasis-related editing tools 

Table D-7-c. Percentage distribution of fourth-grade students using emphasis-related editing tools with 
various frequencies, by how often they took computer-based writing tests with an extended 
constructed-response component: 2012 



How often received computer-based writing tests with an 
extended constructed-response component? 


Number of uses of 
emphasis editing tools 

Never or hardly ever 

1-2 times a month or 
a few times a year 

All students 

All students 

84 

16 

100 

Bold 

0 

60 

61 

60 

1 

17 

17 

17 

2 

8 

7 

8 

>3 

15 

15 

15 

Italics 

0 

69 

70 

69 

1 

14 

12 

13 

2 

7 

6 

7 

>3 

11 

12 

11 

Underline 

0 

70 

71 

70 

1 

15 

13 

15 

2 

6 

7 

6 

>3 

8 

8 

8 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The pattern of using each of the emphasis-related editing tools (i.e., bold, italics, and underline) does 
not differ between students who had the experience of taking computer-based writing tests with an 
extended component and those who did not (table D-7-c), which again, is similar to the language-related 
editing tools. (Absolute value of correlation (r) was smaller than .01 for all three associations.) 
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Revision-related editing tools 

Table D-7-d. Percentage distribution of fourth-grade students using revision-related editing tools with 

different frequencies, by how often they took computer-based writing tests with an extended 
constructed-response component: 2012 



How often received computer-based writing tests with an 
extended CR component? 


Number of uses of 
revision editing tools 

Never or hardly ever 

1-2 times a month or 
a few times a year 

All studi 

All students 

84 

16 

100 

Backspace 

0 

# 

# 

# 

1-100 

54 

55 

55 

101 - 200 

34 

30 

33 

>201 

11 

15 

12 

Delete 

0 

97 

97 

97 

> 1 

3 

3 

3 

Cut 

0 

80 

79 

80 

> 1 

20 

21 

20 

Copy 

0 

87 

85 

86 

> 1 

14 

15 

14 

Paste 

0 

86 

86 

86 

> 1 

14 

14 

14 


# Rounds to zero 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 

Similarly, the pattern of using each of the revision-related editing tools (including backspace, delete, cut, 
copy, and paste) does not differ between students who had the experience of taking computer-based 
writing tests with an extended component and those who did not (table D-7-d). (Absolute value of 
correlation (r) was smaller than .02 for all five associations.) 
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Internet access at home 

Language-related editing tools 

Table D-8-a. Percentage distribution of fourth-grade students using thesaurus editing tools with different 
frequencies, by whether or not they had access to the Internet at home: 2012 


Number of uses of 

Access to the Internet at home? 



thesaurus editing tool 

No 

Yes 

All students 

All students 

23 

77 

100 

Thesaurus replacement 




0 

94 

93 

93 

1 

5 

6 

6 

>2 

1 

1 

1 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The use of thesaurus replacement was not related to students' Internet access at home(r = .03). 

Emphasis-related editing tools 

Table D-8-b. Percentage distribution of fourth-grade students using underline with different frequencies, 
by whether or not they had access to the Internet at home: 2012 


Number of uses of emphasis 

Access to the Internet at home? 



editing tool 

No 

Yes 

All students 

All students 

23 

77 

100 

Underline 




0 

73 

69 

70 

1 

16 

15 

15 

2 

5 

7 

6 

>3 

7 

9 

8 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


Similarly the percentage of students using underline with different frequencies is similar no matter 
students had access to the Internet at home or not, as shown in table D-8-b. 
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Revision-related editing tools 

Table D-8-c. Percentage distribution of fourth-grade students using revision editing tools with different 
frequencies, by whether or not they had access to the Internet at home: 2012 


Number of uses of revision 

Access to the Internet at home? 



editing tools 

No 

Yes 

All students 

All students 

23 

78 

100 

Backspace 

0 

1 

# 

# 

1-100 

58 

53 

55 

101 - 200 

30 

34 

33 

>201 

11 

12 

12 

Delete 

0 

96 

97 

97 

> 1 

4 

3 

3 

Cut 

0 

77 

81 

80 

> 1 

23 

19 

20 

Copy 

0 

86 

86 

86 

> 1 

14 

14 

14 

Paste 

0 

87 

86 

86 

> 1 

13 

14 

14 


# Rounds to zero 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 

As shown in Table D-8-c, there is no substantive difference between fourth-graders who said they had 
Internet access at home and those who said they did not when it comes to using delete, cut, copy, and 
paste editing tools. 
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Using the Internet to get information for writing 

Language-related editing tools 

Table D-9-al. Percentage distribution of fourth-grade students who used spellcheck with different 
frequencies, by how often they looked for information on the Internet to include in their 
writing: 2012 


Number of uses 

How often look for info on the Internet for writing? 


All students 

of spellcheck 
editing tool 

Never or 
hardly ever 

1-2 times a month or a few times a 

year 

At least once a 
week 

All students 

32 

51 


17 

100 

Spellcheck 






0 

26 

23 


25 

24 

1 

15 

13 


16 

14 

2 

12 

11 


12 

11 

3 

9 

11 


9 

10 

>4 

38 

42 


39 

40 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The spellcheck: As table D-9-al indicates, difference in the frequency of looking for information on the 
Internet to include in their writing does not have a substantive relationship with the frequency of using 
the spellcheck editing tool (r = .02). 
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Table D-9-a2. Percentage distribution of fourth-grade students who used thesaurus editing tools with 
different frequencies, by how often they looked for information on the Internet to include in 
their writing: 2012 


Number of uses 

How often look for information on the Internet for writing? 


All students 

of thesaurus 
editing tools 

Never or hardly ever 

1-2 times a month or a 
few times a year 

At least once a week 

All students 

32 

51 


17 

100 

Thesaurus 






0 

60 

58 


56 

58 

1 

28 

30 


30 

29 

>2 

12 

12 


15 

13 

Thesaurus replacement 





0 

94 

93 


93 

93 

1 

5 

6 


6 

6 

>2 

1 

1 


1 

1 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The thesaurus: The pattern of students' using the thesaurus and the thesaurus replacement with 
various frequencies does not differ by how often fourth-grade students looked for information on the 
Internet to include in their writing (table D-9-a2) (r = .03 and .02, respectively), which is similar to the 
use of the spellcheck tool. 
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Emphasis-related editing tools 

Table D-9-b. Percentage distribution of fourth-grade students using emphasis-related editing tools with 
different frequencies, by how often they looked for information on the Internet to include in 
their writing: 2012 



How often look for information on the Internet for writing? 


Number of uses of 
emphasis editing tools 

Never or hardly ever 

1-2 times a month or 
a few times a year 

At least once a 
week 

All students 

All students 

32 

51 

17 

100 

Bold 

0 

62 

60 

57 

60 

1 

15 

17 

18 

17 

2 

8 

8 

9 

8 

>3 

15 

15 

16 

15 

Italics 

0 

70 

69 

67 

69 

1 

13 

13 

14 

13 

2 

6 

6 

7 

7 

>3 

11 

11 

11 

11 

Underline 

0 

71 

69 

69 

70 

1 

15 

15 

16 

15 

2 

6 

7 

7 

6 

>3 

8 

9 

8 

8 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


Table D-9-b indicates that the use of bold, italics or underline does not have a substantive relationship 
with the frequency of how often students looked for information on the Internet for writing. 
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Revision-related editing tools 

Table D-9-c. Percentage distribution of fourth-grade students using some revision-related editing tools 
with different frequencies, by how often they looked for information on the Internet to 
include in their writing: 2012 



How often look for information on the Internet for writing? 


Number of uses of 
revision editing tools 

Never or hardly ever 

1-2 times a month or 
a few times a year 

At least once a 
week 

All students 

All students 

32 

51 

17 

100 

Delete 

0 

97 

97 

95 

97 

> 1 

3 

3 

5 

4 

Cut 

0 

79 

81 

80 

80 

> 1 

21 

19 

20 

20 

Copy 

0 

87 

87 

85 

86 

> 1 

13 

13 

15 

14 

Paste 

0 

87 

86 

85 

86 

> 1 

13 

14 

15 

14 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


In a similar manner, grade 4 students' use of all revision-related editing tools, except the backspace key, 
did not have a substantive relationship with how often students looked for information on the Internet 
for writing. 
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Using the Internet to write to friends or family 


Language-related editing tools 

Table D-10-al. Percentage distribution of fourth-grade students using spellcheck with different 
frequencies, by how often they wrote to friends or family using the Internet: 2012 


Number of uses of 

How often write to friends or family using the Internet? 


All students 

spellcheck editing 
tool 

Never or hardly ever 

1-2 times a month or a 
few times a year 

At least once a 
week 

All students 

44 

32 


24 

100 

Spellcheck 






0 

26 

24 


22 

24 

1 

15 

14 


14 

14 

2 

11 

11 


12 

11 

3 

10 

10 


10 

10 

>4 

38 

41 


42 

40 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The spellcheck: Table D-10-al shows that the percentage of students using the spellcheck tool with 
various frequencies does not differ by how often students reported they write to friends or family using 
the Internet (r < .1). 

Table D-10-a2. Percentage distribution of fourth-grade students using the thesaurus editing tools with 
different frequencies, by how often they wrote to friends or family using the Internet: 2012 


Number of uses of How often write to friends or family using the Internet? 


thesaurus editing 
tools 

Never or hardly ever 

1-2 times a month or a 
few times a year 

At least once a 
week 

All students 

All students 

44 

32 

24 

100 

Thesaurus 

0 

60 

58 

56 

58 

1 

28 

29 

31 

29 

>2 

12 

13 

13 

13 

Thesaurus 

replacement 

0 

94 

93 

92 

93 

1 

5 

6 

7 

6 

>2 

1 

1 

1 

1 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 
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The thesaurus: Like the spellcheck tool, the percentage distribution of students using the thesaurus or 
the thesaurus replacement with various frequencies is similar for students who said they never or hardly 
ever write to friends or family using the Internet and students who said they did it more often (i.e., a 
few times a year or 1-2 times a month or a few times a year, and at least once a week), as shown in table 
D-10-a2 (r < .03 for both variables). 

Emphasis-related editing tools 

Table D-10-b. Percentage distribution of fourth-grade students using emphasis-related editing tools with 
different frequencies, by how often they wrote to friends or family using the Internet: 2012 


Number of uses of 

How often write to friends or family using the Internet? 



emphasis editing 
tools 

Never or hardly ever 

1-2 times a month or a 
few times a year 

At least once a 
week 

All students 

All students 

44 

32 


24 

100 

Bold 

0 

62 

61 


56 

60 

1 

16 

17 


18 

17 

2 

8 

8 


9 

8 

>3 

14 

15 


17 

15 

Italics 

0 

71 

69 


66 

69 

1 

12 

14 


14 

13 

2 

6 

6 


8 

7 

>3 

11 

10 


13 

11 

Underline 

0 

72 

69 


67 

70 

1 

14 

16 


16 

15 

2 

6 

7 


6 

6 

>3 

8 

8 


11 

8 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


Table D-10-b shows that among emphasis-related editing tools, the percentage distribution of students 
using bold, italics and underline at different frequency levels does not differ by how often students said 
that they write to friends or family using the Internet (r < .05 for all three variables). 
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Revision-related editing tools 

Table D-10-c. Percentage distribution of fourth-grade students using revision-related editing tools with 
different frequencies, by how often they wrote to friends or family using the Internet: 2012 



How often write to friends or family using the Internet? 



Number of uses of 
revision editing tools 

Never or hardly ever 

1-2 times a month or a 
few times a year 

At least once a 
week 

All students 

All students 

44 

32 


24 

100 

Delete 

0 

97 

97 


96 

96 

> 1 

3 

3 


4 

4 

Cut 

0 

81 

81 


79 

80 

> 1 

19 

19 


21 

20 

Copy 

0 

88 

86 


83 

86 

> 1 

12 

14 


17 

14 

Paste 

0 

87 

85 


84 

86 

> 1 

13 

15 


16 

14 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


Out of five tools examined, the percentage distribution of students using four tools, i.e. delete, cut, copy 
and paste (r < .06 for these four variables) did not differ by how often students said that they write to 
friends or family using the Internet. 

Writing on the computer for school assignments 

Amount of time writing on computer for school assignment was not substantively related to any of the 
12 editing tools. In other words, the frequency of using the 12 editing tools was very similar across three 
groups with different amount of time writing on computer for school assignments. 

Language-related editing tools 
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Table D-ll-al. Percentage distribution of fourth-grade students using spellcheck editing tools with 
different frequencies, by how much time they spent writing on computer for school: 2012 


Number of uses of 
spellcheck editing tools 

How much time was spent writing on computer for school? 

All students 

None 

10 min 

30 min 

1 hour 

More than 
1 hour 

All students 

34 

19 

26 

13 

8 

100 

Spellcheck 







0 

23 

25 

23 

27 

30 

24 

1 

13 

14 

16 

17 

12 

14 

2 

11 

11 

12 

10 

14 

11 

3 

10 

10 

10 

9 

8 

10 

>4 

42 

40 

40 

37 

37 

40 

Accepted automated spelling corrections 






0 

45 

45 

44 

50 

48 

46 

1 

9 

9 

10 

10 

8 

9 

2 

6 

7 

7 

5 

7 

6 

3 

6 

6 

6 

5 

6 

6 

>4 

34 

33 

33 

31 

31 

33 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The spellcheck: There seems to be a negative relationship between the amount of time spent for 
writing on a computer for school assignments (up to an hour a day) and the frequency of using the 
spellcheck. A higher percentage (30 percent) of the students who said they spend more than one hour 
on computer for school assignments in a day did not use spellcheck at all during the pilot assessment, 
compared to the students who said they spend no time writing on the computer for school assignments 
(23 percent). 

For accepting automated spelling corrections, likewise, a higher percentage (48 percent) of students 
who said they spend more than an hour per day on a computer for school assignments never accepted 
automated spelling corrections during the pilot assessment, compared to students who said they spend 
less than 1 hour per day writing on a computer for school assignments. However, the relationships were 
not substantive (r < .05 for both spellcheck and accepting spelling corrections). 
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Table D-ll-a2. Percentage distribution of fourth-grade students using thesaurus editing tools, by how 
much time they spent writing on computer for school: 2012 



How much time was spent writing on the computer for school? 







More 


Number of uses of 

None 

10 min 

30 min 

1 hour 

than 1 


thesaurus editing tools 





hour 

All students 

All students 

34 

19 

26 

13 

8 

100 

Thesaurus 







0 

59 

59 

57 

58 

57 

58 

1 

27 

28 

31 

31 

30 

29 

>2 

14 

13 

12 

11 

13 

13 

Thesaurus replacement 







0 

93 

93 

94 

94 

91 

93 

1 

6 

7 

5 

6 

7 

6 

>2 

1 

1 

1 

# 

1 

1 


# Rounds to zero. 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 

The thesaurus: No matter how much time students spend on a computer writing for school 
assignments, the percentage distribution of them using the thesaurus or the thesaurus replacement at 
different frequency levels is not different across the 5 time categories (no time, 10 minutes, 30 minutes, 
1 hour, and more than 1 hour), as shown in table D-ll-a2 (r < .01 for both variables). 
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Emphasis-related editing tools 

Table D-ll-b. Percentage distribution of fourth-grade students using emphasis-related editing tools with 
different frequencies, by how much time they spent writing on computer for school:2012 



How much time was spent writing on the computer for school? 


Number of uses of 
emphasis editing tools 

None 

10 min 

30 min 

1 hour 

More than 1 
hour 

All students 

All students 

34 

19 

26 

13 

8 

100 

Bold 

0 

61 

60 

60 

61 

58 

60 

1 

16 

16 

17 

17 

18 

17 

2 

8 

8 

8 

8 

8 

8 

>3 

15 

15 

15 

15 

16 

15 

Italics 

0 

68 

71 

68 

70 

70 

69 

1 

13 

12 

14 

14 

13 

13 

2 

7 

7 

7 

6 

5 

7 

>3 

11 

10 

11 

10 

12 

11 

Underline 

0 

70 

71 

69 

72 

69 

70 

1 

15 

15 

15 

15 

17 

15 

2 

6 

7 

7 

5 

6 

6 

>3 

9 

8 

9 

8 

8 

8 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


Table D-ll-b shows the findings regarding three emphasis-related editing tools (bold, italics and 
underline): the percentage distribution of students using the tools at various frequency levels is not 
different by the amount of time that students spent writing on a computer for school assignments per 
day. (Absolute value of correlation (r) was smaller than .01 for all three correlations). 
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Revision-related editing tools 

Table D-ll-c. Percentage distribution of fourth-grade students using revision-related editing tools with 
different frequencies, by how much time they spent writing on computer for school: 2012 



How much time was spent writing on the computer for school? 


Number of uses of 
revision editing tools 

None 

10 min 

30 min 

1 hour 

More than 1 
hour 

All students 

All students 

34 

19 

26 

13 

8 

100 

Backspace 

0 

# 

# 

# 

# 

# 

# 

1-100 

59 

54 

51 

53 

51 

54 

101 - 200 

31 

33 

36 

34 

36 

33 

>201 

10 

12 

13 

12 

13 

12 

Delete 

0 

97 

96 

97 

96 

96 

97 

> 1 

3 

4 

3 

4 

4 

3 

Cut 

0 

80 

80 

82 

80 

76 

80 

> 1 

20 

20 

18 

20 

24 

20 

Copy 

0 

87 

85 

87 

85 

83 

86 

> 1 

13 

15 

13 

15 

17 

14 

Paste 

0 

86 

86 

86 

86 

84 

86 

> 1 

14 

14 

14 

14 

16 

14 


# Rounds to zero. 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 

Table D-ll-c shows that the percentage distribution of students using delete, cut, copy, and paste at 
various frequency levels is not different by the degree of frequency of spent time writing on a computer 
for school assignments (absolute value of r was smaller than .1 for all five variables). However, students 
who said they spend no time writing on a computer for school assignments are more likely to use 
backspace 1-100 times, compared to students who said they spend 30 minutes per day writing on the 
computer for school assignments (59 percent vs. 51 percent). 
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3. The relationship between writing performance and prior exposure 
related to writing on computer 

Instruction on keyboarding and word processing 

Table D-12. Percentage and average raw scores of fourth-grade students whose teachers and staff did or 
did not spend time instructing students on keyboarding and word processing, by performance 
level: 2012 



Percentage 

Average raw scores 

Score difference (Yes - No) 

Performance level 

No 

Yes 

No 

Yes 

Low performing 

14 

86 

1.38 

1.40 

0.02 

Middle performing 

13 

87 

2.85 

2.89 

0.04 

High-performing 

10 

90 

4.72 

4.74 

0.02 


NOTE: Detail may not sum to totals because of rounding. Average raw score is reported on the scale of 1 to 6. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Education Progress (NAEP), 2012 Writing Pilot Assessment. 


The score difference between those who received instruction on keyboarding and word processing and 
those who did not was 0.2 with an effect size of .17, suggesting a small effect. This effect of score 
difference, however, disappears when examined by performance level (table D-12). The percentage of 
students receiving keyboarding and word processing instructions was also not substantively different 
across performance levels ( r= .04). 

Computer-based writing tests with an extended constructed-response component 

Table D-13. Percentage and average raw scores of fourth-grade students who did or did not take 
computer-based writing tests with an extended constructed-response component, by 
performance level: 2012 



Percentage 

Average raw scores 

Score difference (Yes -No) 

Performance level 

No 

Yes 

No 

Yes 

Low performing 

83 

17 

1.39 

1.38 

-0.02 

Middle performing 

85 

15 

2.88 

2.87 

-0.01 

High-performing 

84 

16 

4.72 

4.83 

0.10 


NOTE: Detail may not sum to totals because of rounding. Average raw score is reported on the scale of 1 to 6. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Education Progress (NAEP), 2012 Writing Pilot Assessment. 


The average score of those who had never or hardly ever taken computer-based writing tests with an 
extended constructed-response component was about 0.1 or lower point different from the score of 
those who had taken such tests for each performance group. These score differences were not 
substantive, suggesting no potential effects of prior experience of taking computer-based writing tests 
with an extended constructed response component on writing performance. 
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Language-related editing tools 

Table E-l. Percentage distribution of fourth-grade students who used thesaurus editing tools with various 
frequencies, by mode preference to take a writing test: 2012 


Mode preference to take a writing test 

Number of uses of thesaurus editing tool Paper Computer Does not matter 

Thesaurus replacement 

0 96 91 94 

1 3 7 5 

>2 111 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 

As table E-l shows, there were no substantive differences in how frequently the thesaurus replacement 
editing tool was used by students who preferred to take a writing test on paper, on the computer, or 
does not matter. Additionally, most of the fourth-graders did not use the editing tool during the pilot 
assessment. 

Emphasis-related editing tools 

Table E-2. Percentage distribution of fourth-grade students who used the underline tool with various 
frequencies, by mode preference to take a writing test: 2012 


Mode preference to take a writing test 


Number of uses of emphasis editing tool 

Paper 

Computer 

Does not matter 

derline 




0 

73 

68 

71 

1 

14 

15 

16 

2 

5 

7 

6 

>3 

7 

10 

7 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


Different than the other two emphasis-related editing tools (bold and italic), the distribution of students 
with various frequencies of using the underline editing tool is not substantively different across the 
preferred modes of writing: paper, computer or does not matter (see table E-2). 
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Revision-related editing tools 

Table E-3. Percentage distribution of fourth-grade students who used several revision-related word 
processing tools with various frequencies, by mode preference to take a writing test: 2012 


Number of uses of revision 


Mode preference to take a writing test 


editing tools 

Paper 

Computer 

Does not matter 

Delete 

0 

97 

96 

96 

> 1 

3 

4 

4 

Cut 

0 

81 

79 

81 

> 1 

19 

21 

19 

Copy 

0 

88 

85 

87 

> 1 

12 

15 

13 

Paste 

0 

90 

84 

86 

> 1 

10 

16 

14 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


The percentage distribution of students by the frequency of use of the delete, cut, copy, and paste 
editing tools was similar across the three preferred modes of writing (table E-3). 
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Relationship between prior exposure and preference for mode of 
writing 

Instruction on keyboarding and word processing 

Table E-4. Percentage distribution of fourth-grade students with different preference for the mode of a 
writing test, by whether or not they received instructions on keyboarding and word 
processing: 2012 


Flow do you prefer to take a writing test? 

Time spent on keyboarding and word processing 
No 

? 

Yes 

All students 

Paper 

17 

18 

18 

Computer 

48 

46 

46 

Does not matter 

34 

36 

36 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


As shown in table E-4, whether or not students had received instruction on keyboarding and word 
processing is not related to the pattern of their preference for the mode of writing. For example, a 
similar percentage of students preferred to take the writing test on the computer no matter they had 
had instructions on keyboarding and word processing or not (46 percent vs. 48 percent). 

Computer-based writing assignments 

Table E-5. Percentage distribution of fourth-grade students with different preference for the mode of a 
writing test, by how often they received computer-based writing assignments: 2012 



Flow often received computer-based writing assignments? 

All students 

Flow do you prefer to 
take a writing test? 

Never or hardly 
ever 

1-2 times a month or a 
few times a year 

At least once a 
week 

Paper 

18 

18 

17 

18 

Computer 

47 

46 

45 

46 

Does not matter 

35 

36 

38 

36 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


Regardless of the frequency of receiving computer-based writing assignments, a similar percentage of 
students preferred each mode of writing. For example, 47 percent of the students who never or hardly 
ever received those assignments, 46 percent of those who received those assignments 1-2 times a 
month or a few times a year, and 45 percent of those who received computer-based assignment at least 
once a week preferred writing on the computer. 
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Computer-based writing tests with an extended constructed-response 

Table E-6. Percentage distribution of fourth-grade students with different preference for the mode of a 
writing test, by how often they took computer-based writing tests with an extended 
constructed-response component : 2012 


How often take computer-based writing tests with an 



extended CR component? 


All students 

How do you prefer to take a 
writing test? 

At least once a week, 1-2 times 
Never or hardly ever a month, or a few times a year 

Paper 

18 

16 

18 

Computer 

46 

46 

46 

Does not matter 

35 

38 

36 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


As table E-6 shows, the distribution of students by their preference for the mode of a writing test is not 
different with regard to how often students have taken computer-based writing tests with an extended 
constructed response component. The biggest difference was observed in the “Does not matter" 
category. While 35 percent of students who had never or hardly ever taken computer-based writing 
tests with an extended CR component responded that mode of writing "does not matter", 38 percent of 
those who had some such experience responded that it "does not matter". The difference of 3 percent, 
however, is not substantive. 

Using the Internet to get information for writing 

Table E-7. Percentage distribution of fourth-grade students with different preference for the mode of a 
writing test, by how often they looked for information on the Internet to include in their 
writing: 2012 


How do you prefer to 
take a writing test? 

How often look for information on the Internet to include in writing? 

All students 

Never or hardly ever 

1-2 times a month or a 
few times a year 

At least once a week 

Paper 

20 

17 

16 

18 

Computer 

44 

47 

49 

46 

Does not matter 

37 

36 

35 

36 


NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Education 
Progress (NAEP), 2012 Writing Pilot Assessment. 


Table E-7 shows that the frequency with which students looked for information on the Internet to include 
in their writing is not substantively related to their preferred mode to take a writing test. For example, 49 
percent of students who said that they look for information on the Internet to include in their writing at 
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least once a week preferred writing on the computer, and the number is similar (44 percent) for students 
who never or hardly ever searched for such information. 
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Figure F-l: Boxplot for the distribution of word count, by raw score: 2012 



Appendix F. Figures 


Figure F-2: Boxplot for the distribution of word count, by the number of automated spelling corrections 
accepted: 2012 


Word Count 

700 


600 

500 

400 

300 

200 

100 

0 


4 or more 
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NOTE: Plus sign in the box indicates average word count for the subgroup. Florizontal line in the box indicates 
median word count for the subgroup. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Writing Pilot Assessment. 
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Figure F-3a: Boxplot for the distribution of word count, by number of uses of backspace: 2012 
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NOTE: Plus sign in the box indicates average word count for the subgroup. Florizontal line in the box indicates 
median word count for the subgroup. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Writing Pilot Assessment. 
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Figure F-3b: Boxplot for the distribution of word count, by number of uses of cut: 2012 


Word Count 

700 

600 
500 
400 
300 


200 


100 


Number of uses of cut 


1 or more 


NOTE: Plus sign in the box indicates average word count for the subgroup. Florizontal line in the box indicates 
median word count for the subgroup. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Writing Pilot Assessment. 
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Figure F-4: Boxplot for the distribution of word count, by access to the Internet at home: 2012 
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NOTE: Plus sign in the box indicates average word count for the subgroup. Florizontal line in the box indicates 
median word count for the subgroup. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Writing Pilot Assessment. 
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Figure F-5: Boxplot for the distribution of word count, by the frequency of the use of the Internet to look 
for information to include in their writing: 2012 
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NOTE: Plus sign in the box indicates average word count for the subgroup. Florizontal line in the box indicates median 
word count for the subgroup. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 2012 Computer-Based Writing Pilot Assessment. 
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Figure F-6: Boxplot for the distribution of word count, by the frequency of the use of the Internet to write 
to friends or family: 2012 
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indicates median word count for the subgroup. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2012 Computer-Based Writing Pilot Assessment. 
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Figure F-7: Boxplot for the distribution of word count, by mode preference on writing test: 2012 
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